Case l:14-cv-00857-TSC Document 70-50 Filed 01/22/16 Page 1 of 127 


EXHIBIT 51 



Case l:14-cv-00857-TSC Document 70-50 Filed 01/22/16 Page 2 of 127 


IN THE UNITED STATES DISTRICT COURT 
FOR THE DISTRICT OF COLUMBIA 


AMERICAN EDUCATIONAL RESEARCH Case No. 1 : 14-cv-00857-TSC-DAR 

ASSOCIATION, INC., AMERICAN 

PSYCHOLOGICAL ASSOCIATION, INC., and 

NATIONAL COUNCIL ON MEASUREMENT IN 

EDUCATION, INC., 

Plaintiffs/Counter-defendants, 

v. 

PUBLIC.RESOURCE.ORG, INC., 

Defendant/Counterclaimant. 


EXPERT REPORT OF JAMES R. FRUCHTERMAN 



Case l:14-cv-00857-TSC Document 70-50 Filed 01/22/16 Page 3 of 127 


Table of Contents 

Introduction 1 

Background and Qualifications 2 

What Does Accessibility Mean for a Person Who is Blind? 3 

Locating an Accessible Version of the 1999 Standards 5 

Testing the Public. Resource. Org Website's Accessibility 7 

Making the 1999 Standards Accessible Today 8 

The Public.Resource.Org Version of the 1999 Standards 9 

Confirming Accessibility for People Who Are Blind 9 

The Archive.org Version of the 1999 Standards 11 

Conclusion 12 


Case l:14-cv-00857-TSC Document 70-50 Filed 01/22/16 Page 4 of 127 


Introduction 

As an expert in accessibility of written materials for people who have disabilities that 
affect using standard print (people who are print disabled), I have been retained by 
Public. Resource. Org to evaluate the accessibility of certain content that had been available on 
the website of the defendant in this case. As someone dedicated to improving accessibility for 
the benefit of people with disabilities and in the public interest, I agreed to evaluate the 
accessibility to people who are blind of this specific commonly used standard document. 

This expert report is a summary of certain opinions that I intend to give, if asked, at trial 
regarding the accessibility of specific documents to people who are blind or print disabled. This 
report also states the bases for my opinions, and it discloses the data or other information 
considered in forming those opinions. I reserve the right to change or supplement this report if 
additional evidence comes to my attention, and to prepare demonstratives and/or exhibits to 
illustrate or explain my opinions, as appropriate. 

A copy of my curriculum vitae, including a list of my publications and presentations, is 
Exhibit A to this report. I provide my expertise in this case pro bono, and I am not receiving 
compensation for my time researching, writing this report, or testifying. I previously served as 
an expert in The Authors Guild, Inc. et al. v. HathiTrust, et al., Case No. 1 : 1 l-cv-0635 1-HB 
(S.D.N.Y.) (case filed September 12, 201 1) and I am serving as an expert in American Society of 
Testing and Materials, et al. v. Public.Resource.Org, Case No. l:13-cv-01215-TSC-DAR, 
although I have not testified in either case. I have not given deposition or trial testimony in the 
past four years. 


1 


Case l:14-cv-00857-TSC Document 70-50 Filed 01/22/16 Page 5 of 127 


Background and Qualifications 

I serve as Founder and Chief Executive Officer of Benetech, a nonprofit dedicated to 
creating new technology solutions that serve humanity and empower people to improve their 
lives. In 1980 I earned a B.S. in Engineering and an M.S. in Applied Physics from California 
Institute of Technology. I co-founded Calera Recognition Systems in 1982. Calera developed 
optical character recognition (OCR) technology that allowed computers to read virtually all 
printed text. 

In 1989, 1 founded Arkenstone, a nonprofit social enterprise, which produced reading 
machines for the print disabled community based on the Calera technology, and was at one time 
the largest maker of affordable reading systems for the blind. The Arkenstone product line was 
sold in 2000 and the resulting capital funded the next phase of Arkenstone under its new name, 
Benetech. I have been the CEO of Benetech/ Arkenstone since 1989. 

I have served on three U.S. federal government advisory committees for disability issues: 
the Section 255 Telecommunications Access Advisory Committee, the Section 508 Electronic 
Infonnation and Technology Access Advisory Committee, and the Advisory Commission on 
Accessible Instructional Materials in Postsecondary Education for Students with Disabilities. 

I have received numerous other awards and recognition for my work making print materials 
accessible to people who are blind or otherwise print disabled. In 2006 I received a MacArthur 
Fellowship. I was named an Outstanding Social Entrepreneur in 2003 by the Schwab 
Foundation and have frequently participated in the World Economic Forum Annual Meetings in 
Davos, Switzerland. Benetech received the Skoll Award for Social Entrepreneurship under my 
leadership. I also received the Migel Medal from the American Foundation for the Blind, the 
Robert F. Bray Award from the American Council of the Blind, and the American Library 
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Association’s Francis Joseph Campbell Award in recognition of my successful efforts to make 
literary works more accessible to people who are blind or visually impaired. 

What Does Accessibility Mean for a Person Who is Blind? 

Accessibility is usually defined in a functional way: can a person with a disability 
independently access the same information and perform the same tasks as a person without a 
disability? When it comes to accessing materials traditionally available as print, such as 
standards, there are many groups of print disabilities. The most severe is blindness, where a 
person cannot perceive the printed text at all. The next is vision impairment, where a person 
generally cannot perceive the text directly or with corrective lens, but may be able to use 
magnifiers of different types to read the text. Another group is learning disabilities that interfere 
with reading, such as dyslexia. A closely related group of disabilities involve brain injuries that 
affect reading or the retention of material read. Another group is physical disabilities that 
interfere with the holding or seeing of books or the turning of pages. 

In this report, I focused on the accessibility challenges that would be experienced by 
blind people, because they are generally the most severe print disabilities. The other groups of 
people with print disabilities use similar technologies to access print (such as having it read 
aloud), and experience similar challenges as blind people. In the accessibility field, it is 
generally understood that if you make infonnation accessible to a blind person, it will probably 
also meet the accessibility needs of the great majority of people with other print disabilities. 

The most common technology used by a blind person for accessibility is called a screen 
reader. As the name suggests, a screen reader is a program that runs on a personal computer or a 
smartphone that reads the information on the screen aloud (using a computer-synthesized voice) 
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to a blind person. The screen reader runs “on top of’ other programs, figuring out not only what 
text is on the screen, but also the controls that are displayed: items such as buttons, menus, text- 
entry boxes and the like. Because of the amount of information on a complete screen, and its 
complexity, blind people need to be able to focus on the most important information so that they 
do not waste time listening to everything on the screen. 

For the purpose of this report, measuring the accessibility of standards, I am assuming 
that the blind user is using a screen reader on top of a web browser or word processor program 
on a personal computer. Based on the information the screen reader can glean from the pages 
displayed on the screen, can a blind person locate the standard and read it? 

The accessibility tasks I tested were designed to assess whether a blind user with basic 
assistive technology skills could perform the same kind of tasks one might expect a user without 
a disability to perform in accessing a given standard, without requiring the intervention of a third 
party. This functional approach is the most common method of assessing accessibility. 

The specific tasks I investigated were: 

• Could a blind user with basic assistive technology skills independently access a 
specific standard of interest? 

• Could a blind user independently read the entire standard using assistive technology? 

• Could a blind user independently navigate to a specific place in the standard and read 
the content in that place? 

• Could a blind user independently do a full text search and find specific mentions of 
terms of interest? 
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I conducted these tests on a standards document that was represented to me as having been 
available on the Public. Resource. Org website. I primarily used the Window-Eyes screen reading 
software and the ABBYY FineReader optical character recognition software to perform my tests. 

Locating an Accessible Version of the 1999 Standards 

I was asked to review the accessibility of the 1999 edition of The Standards for 
Educational and Psychological Testing (hereafter, the “1999 Standards”) for people who are 
blind or otherwise print disabled. The first step in determining the accessibility of a document is 
to try to locate a version of the 1999 Standards that would be accessible to people who are blind 
or have print disabilities. I attempted to locate an accessible version of the 1999 Standards 
through two separate avenues: by searching the catalogs of the main libraries that serve people 
with print disabilities, and also by doing a standard Google search to try to locate an electronic 
version of the 1999 Standards. From my work with people who are blind or print disabled, I 
know that this would be the typical procedure that people who are blind or print disabled would 
perform when looking for an accessible version of a document. 

The four main libraries that serve people with print disabilities are the American Printing 
House for the Blind, Bookshare (which I founded), Learning Ally, and the National Library 
Service for the Blind and Physically Handicapped, Library of Congress. I performed a thorough 
search of all four of these catalogs and found that the 1999 Standards were not available through 
any of these resources, either in an electronic form, or in mail-delivery braille or audio recording. 

I then performed a Google search to attempt to locate an electronic version of the 1999 
Standards online. I was unable to find an electronic version of the 1999 Standards online, but I 
did locate a used print version for sale on Amazon.com. I have been informed by counsel for 
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Public. Resource. Org that although Public.Resource.Org previously hosted an electronic version 
of the 1999 Standards on its website, it had been taken down during the course of this litigation. 
From my research I believe that a version of the 1999 Standards that is accessible to people who 
are blind or print disabled is currently unavailable to the public. 

The unavailability of a version of the 1999 Standards that is accessible to people who are 
blind or print disabled is problematic because the 1999 Standards are important references for 
those making tests that are accessible to students who are print disabled, as well as those 
impacted by these tests. For instance, the 1999 Standards were referred to in several works 
concerning test accessibility for blind students, specifically: Test Access: Making Tests 
Accessible for Students with Visual Impairments : A Guide for Test Publishers, Test Developers, 
and State Assessment Personnel, Second Edition, by Carol B. Allman, Ph.D., published by the 
American Printing House for the Blind ( Exhibit C) , and an online resource published by the 
American Foundation of the Blind, Building Assessment Initiatives for Schools: Guidelines to 
Support the Contract Development Process Between Test Publishers and States (Exhibit D) . As 
an expert in the field, this means that the 1999 Standards are important references today for those 
making tests accessible to students with disabilities such as blindness. This also means that it is 
an important resource to any students or other individuals with print disabilities that want to 
assess compliance with the 1999 Standards. The unavailability of the 1999 Standards means that 
some of those who are most impacted, people who are blind or print disabled, are unable to 
independently access the 1999 Standards. 
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Testing the Public.Resource.Org Website's Accessibility 

Because the 1999 Standards are no longer hosted on the Public.Resource.Org website 
during the course of this litigation, I was not able to locate the full text of the 1999 Standards on 
the Public.Resource.Org website while performing my Google search referenced above. 
However, searching the terms “1999 Standards for Educational and Psychological Testing” on 
Google (for me) shows the page where the 1999 Standards had been located on the 
Public.Resource.Org website, located in the first page of links in the search results. However, 
the file I found there was a placeholder noting the voluntary takedown of the file. I have also 
searched for other standards that Public.Resource.Org has posted on its website, such as NFPA 
101-2000, and I have found that it would be relatively easy for a person who is blind or print 
disabled to use screen reader software and perform a Google search to locate a standard if it was 
available on Public.Resource.Org’s website. Therefore, when the 1999 Standards had been 
hosted on the Public.Resource.Org website, a person who is blind would have been able to locate 
the 1999 Standards through a simple Google search, with the assistance of screen reader 
software. 

The Public.Resource.Org website has no required sign-up procedure. It is possible to go 
directly to a specific standard either by using a direct weblink or by navigating the text-oriented 
website. This is important because sign-up procedures can often have the effect of preventing 
people who are blind or print disabled from accessing certain parts of websites due to the fact 
that many sign-up procedures use unlabeled buttons or other elements that screen reader software 
cannot read. Therefore, a person who is blind or print disabled would have been able to locate a 
version of the 1999 Standards on the Public.Resource.Org website when it was still hosted there, 
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and that person would then have been able to gain access to that electronic version of the 1999 
Standards. 

Making the 1999 Standards Accessible Today 

Because an accessible version of the 1999 Standards is not currently available, if a blind 
person needed to have an accessible version of the 1999 Standards, they would need to create it 
themselves or request that their employer, educational institution, or a specialized library for the 
blind create it. Generally, most blind people themselves do not have the ability to convert books. 
Some blind people have their own home scanners, and if they purchased a used copy online, 
would be able to scan the 1999 Standards page by page on a home scanner, which would take at 
least two hours of labor, and then perform optical character recognition on the title. Optical 
character recognition is the process by which a computer converts images of printed text into 
machine-encoded text that can be read aloud by a screen reader. If the scanning quality wasn’t 
very good, significant numbers of errors would be introduced through the optical character 
recognition process. The resulting word processor file of recognized text could then be read 
using a screen reader. 

If Bookshare were to make the 1999 Standards accessible to a blind person, we would 
purchase a used copy of the printed version, chop off the bindings and then process it through a 
high speed scanner to obtain a high quality scan of the book in less than fifteen minutes. We 
would then perform optical character recognition on the image scans of all of the pages of the 
book, which typically creates a Microsoft Word file version of the text, and then send it to an 
outside service (or a volunteer) to have it proofread, correcting errors introduced by the 
limitations of optical character recognition. Public. Resource. Org has already performed the 
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great majority of the most expensive and time consuming steps needed to create an accessible 
version of this document, specifically purchasing a print version of the title, waiting a few days 
to receive it, chopping off the binding and scanning it with a high speed production scanner, or 
utilizing a library-grade nondestructive book scanner. This is a valuable contribution to anyone, 
individual or organization, that wanted to ensure that the 1999 Standards are accessible to people 
who are blind or print disabled, if that file were still available. 

The Public.Resource.Org Version of the 1999 Standards 

I was supplied with a version of the 1999 Standard in PDF format. It was represented to 
me that this file had been available online at the Public.Resource.Org website. I examined the 
file, and found it to be a high quality image scan of the 1999 Standards. If the file was still 
online, this would have meant that a blind person wanting to have an accessible version of the 
1999 Standards would be able to do so by performing optical character recognition on the 
Public.Resource.Org image file, creating an accessible text version of the 1999 Standards in 
minutes. 

Confirming Accessibility for People Who Are Blind 

I then performed the steps of taking the Public.Resource.Org version of the 1999 
Standards and making it accessible, while using Window-Eyes screen reading software to read 
the words on the computer screen aloud. For the version of the 1999 Standards that was on the 
Public.Resource.Org website, I used ABBYY FineReader optical character recognition software 
to recognize page images, and it converted those pages into a Microsoft Word document. In 
addition, the process of using the ABBYY software and reading the document was something a 
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blind person could do independently using Window-Eyes software to perform the tasks in an 
accessible way, because the program speaks the menus and converted text aloud. Because the 
image scan by Public. Resource. Org was high quality, there were few optical character 
recognition errors. In addition, I also tested a typical page image from the Public. Resource. Org 
version using the website Free Online OCR (http://www.onlineocr.net/) , and confirmed that it 
also recognized the text well. In my opinion, most of the OCR solutions that would be available 
to people who are blind should be able to convert this image PDF document into accessible text. 
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I then examined in Microsoft Word several pages of the standard as processed by 
ABBYY FineReader, and confirmed that Window-Eyes could read the text aloud in logical 
reading order. I also successfully perfonned full text searches on a key word, a standards 
number, and a page number, using Window-Eyes. 
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My tests therefore indicated that a blind person using a screen reader would be able to 
perform all of the functional tasks: reading the entire standard, navigating to a specific place in 
the standard, or searching on key terms. Because the text is provided in a standard format, such 
as Microsoft Word, a blind person is able to listen to the text, or access it using a digital braille 
device. This kind of text content is also highly accessible to people with other print disabilities 
and the assistive technology they use to access print. For example, people with low vision or 
with dyslexia often use a screen reader to read text aloud. 


The Archive.org Version of the 1999 Standards 

I was also supplied with a version of the 1999 Standard in TXT (text) format, by a staff 
person at the Internet Archive, operator of the Archive.org website. It was represented to me by 
this person that this file, aera.standards.l999_djvu.txt, had been available online at the 
Archive.org website. According to the Internet Archive’s “Derivatives” page located at 


11 



Case l:14-cv-00857-TSC Document 70-50 Filed 01/22/16 Page 15 of 127 


https :// archive. org/help/derivatives .ptip, when a PDF file is uploaded to the Internet Archive 
website, the website automatically creates derivative file types that are also accessible on that 
website, including TXT format. The deposition testimony of Christopher Butler from the 
Internet Archive, as well as the deposition testimony of Carl Malamud from Public.Resource.Org 
indicate that when Public Resource uploaded the PDF file of the 1999 Standards to the Internet 
Archive website, the Internet Archive automatically created this text file of the 1999 Standards, 
which was publicly accessible on the Internet Archive website. 1 I examined the file, and found it 
to be a text version of the 1999 Standards, preceded by informational material about the Internet 
Archive in HTML format. It appeared to me that the text version had been created by optical 
character recognition, because there were a few uncorrected errors typical of that process. 

As established in my discussion of the 1999 Standards on the Public.Resource.Org 
website above, once the 1999 Standards are available in an electronic text format, a blind person 
using a screen reader would be able to perfonn all of the functional tasks: reading the entire 
standard, navigating to a specific place in the standard, or searching on key terms. I confirmed 
that this was the case with the aera.standards.l999_djvu.txt file. Because the text is provided in 
a standard and compatible format, a blind person is able to listen to the text, or access it using a 
digital braille device. This kind of text content is also highly accessible to people with other 
print disabilities and the assistive technology they use to access print. 

Conclusion 

I was asked to review the accessibility of the 1999 edition of The Standards for 
Educational and Psychological Testing. I was unable to find an accessible version of the 

1 Deposition of Christopher Butler of the Internet Archive, December 2, 2014, at pp. 48-49, 87, 102-105; 
deposition of Carl Malamud of Public.Resource.Org, May 12, 2015, atpp. 281-284. 
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document online. If the document provided to me by Public.Resource.Org had been online on 
their website, I believe a blind person of ordinary technical skill would have been able to 
independently use that document and commonly available optical character recognition 
technology to create an accessible version of the 1999 Standards, and carry out reading and 
reference tasks similar to those a person without a disability would be able to do with a print 
version of the standard. If the document provided to me by Archive.org had been online on their 
website, I believe a blind person of ordinary technical skill would have been able to 
independently use that document directly to carry out reading and reference tasks similar to those 
a person without a disability would be able to do with a print version of the standard. 


Dated: June 13, 2015 

7 / / 7 ' 

James R. Fruchterman 
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James R. Fruchterman 

Founder and CEO 
Benetech 


Education 

■ California Institute of Technology 
B.S. Engineering, 1976-80 

M.S. Applied Physics, 1978-80 

■ Stanford University, 1980-81 

Ph.D. Studies in Electrical Engineering 

Professional Experience 

■ CEO and Founder, 2015-present 

President, CEO, Chairman, Founder, 2000-2014 
Benetech (name changed from Arkenstone in 2000) 
Palo Alto, California 

■ President, CEO, Chairman, Founder, 1989-2000 
Arkenstone, Inc. 

Moffett Field, California 

■ Director, 1989-present 

Vice President Finance, CFO, 1989-2004 
President & CEO, Founder, 1989-95 
RAF Technology, Inc. 

Palo Alto, California and Redmond, Washington 

■ Vice President, Marketing, 1987-89 
Founder, Vice President, Finance, 1982-88 
Calera Recognition Systems, Inc. 

Santa Clara, California 

■ Prior engineering positions with: 

■ Phoenix Engineering, Inc. 

■ G.C.H., Inc. 

■ IBM T.J. Watson Research Center 

■ General Motors Company 

■ NASA — Jet Propulsion Laboratory 

■ Fermi National Accelerator Laboratory 
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Publications 

■ Technology Serving Humanity (chapter). In Schultz, R. (editor) Creating Good Work . Palgrave 
Macmillan, February 2013 

■ Guest Editor’s Page. AFB Journal of Visual Impairment & Blindness, October-November 

2012 

■ An Interview With Technology Guru George Kerscher, AFB Journal of Visual Impairment 
& Blindness, October-November 2012 

■ For Love or Lucre. Stanford Social Innovation Review, Spring 201 1 

■ Developing Information Technology to Meet Social Needs . In Innovations , MIT Press, 2008 

■ Accessing Books and Documents, a chapter in the book, Assistive Technology for Vision- 
Impaired and Blind People . Springer Verlag 2008 

■ Everyone Deserves Access to Technology. OpEd in The Sacramento Bee by Jim Fmchterman 
and Gregg Vanderheiden, June 17, 2007 

■ Document Recognition Serving People With Disabilities, Proc. SPIE 6500, International 
Society for Optics and Photonics, 2007 

■ Pattern Recognition Technology Helps Disabled People Access Books. SPIE Newsroom, 
International Society for Optics and Photonics, May 14, 2007 

■ Nothing Ventured Nothing Gained. Addressing the Critical Gaps in Risk-Taking Capital for 

Social Enterprise, by Jed Emerson, Tim Freundlich and Jim Fmchterman, published by 
Oxford Said Business School, 2006 

■ Build Great Companies. Then Help Build a Great World. OpEd in The San Jose Mercury News, 
November 13, 2006 

■ Comments on Accessibility of Google Print and Google’s Library Project, white paper, 
February 2005 

■ Technology Benefiting Humanity , published in the Association for Computing Machines 
Ubiquity magazine, March 2004 

■ The Power of Technology Social Enterprises, published in the N-TEN forecast series, 
February 2004 

■ In the Palm of Your Hand: A Vision of the Future of Technology for People with Visual 

Impairments, published in the American Foundation for the Blind’s Journal of Vision 
Impairment and Blindness . October 2003 

■ The Chafee Amendment: Improving Access to Information , published in Information 
Technology and Disabilities , a journal published by Equal Access to Software and Information 
(EAST, co-authored with Bookshare Senior Product Manager Alison Lingane, October 2003 

■ The Soundproof Book: Exploration of Rights Conflict and Access to Commercial EBooks 

for People with Disabilities, published in First Monday , co-authored with George Kerscher, 
the International Project Manager of die DAISY Consortium . May 2002 


Bookshare. Books without Barriers , at the Closing the Gap conference, Minneapolis, MN, 
October 2001 
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■ Two presentations given at the IT Accessibility 2001 Conference, May 2001 at the National 
Institute of Standards and Technology 

■ I Dream of Software 

■ The Business Case for Adaptive Technology 

■ Humanizing the Voice of the Machine, with Prof. Mari Ostendorf (University of 
Washington), Annual Meeting of the American Association for the Advancement of the 
Machine, Boston, MA, February 2000 

■ The Many Facets of Open Book: Ruby Edition. California State University, Northridge 
(CSUN), 15th Technology and Persons with Disabilities Conference, March 2000 

■ Corporate Responsibility for Adaptive Technology. California State University, Northridge 
(CSUN), 14th Technology and Persons with Disabilities Conference, March 1999 

■ Developing Partnerships for Assistive and Universally Designed Technology for Persons 

with Disabilities . Testimony before United States House of Representatives, Committee on 
Science, Subcommittee on Technology, August 4, 1998 

■ Access to Maps and Location Information through Virtual Reality Techniques and GPS 
Satellite Receivers, 3rd International Technical Aids Seminar, Tokyo, Japan, July 1994 

Invited T alks 

■ “ Innovation in Atnerica: The Role of Technology ” August 1, 2013, Testimony before U.S. 
House of Representatives, Judiciary Committee’s Subcommittee on Courts, Intellectual 
Property, and the Internet. 

■ “Social Change at Scale - That’s Innovation!” May 2012, TEDxSanJoseCA 2012, San Jose, 
CA. 

■ “T he Power of Failure, People and Karma Rankins, ’’ May 20, 2012, Commencement speech, St. 
Mary's College, Moraga, CA. 

■ “Raising the Floor, ” October, 201 1 , Keynote Speech, Association for Education and Rehabilitation 
of the Blind and Visually Impaired Conference, Cleveland OH. 

■ Keynote speech, IEEE Sections Congress, August 20 1 1 , San Francisco, CA. 

■ “ Making the Book Tmly Accessible. ” Tools of Change Conference, New York, NY, 201 1 
Keynote Speech 

■ UBS-Ashoka Visionaris Award, Keynote Speech, Social Entrepreneur of the Year Award, 
Mexico City, Mexico, September, 2010 

■ A series of three invited speeches on Bookshare and accessible books, in Tokyo, Shizuoka 
and Osaka, Japan, February, 2009 

■ Keynote Speech, Social Enterprise World Forum, Edinburgh, Scotland, September, 2008 

■ “Raising the Floor: Providing Accessible Technology and Content to Every Person with a 
Disability on the Planet,” International Conference on Computers Helping People with 
Special Needs, Linz, Austria, July, 2008 Keynote Speech 

■ “Raising the Floor,” CSUN Conference on Technology and Persons with Disabilities, 
March, 2008 Keynote Speech 
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■ Extensive speaking engagements to students about technology serving people with 
disabilities. Have done invited talks at: 

■ Stanford University 

■ University of California at Berkeley 

■ Brigham Young University 

■ University of the Pacific 

■ Santa Clara University 

■ California Institute of Technology 

■ San Jose State 

■ University of California at Santa Cruz 

■ University of California at Davis 

■ Loyola Marymount University 

■ Pepperdine University 

■ University of Washington 

■ Columbia University 

■ Harvard University 

■ University of Geneva 

■ Oxford University 

■ Inflection Point Opportunities in Social Investment, Closing Keynote for the UBS 
Philanthropy Fomm, Lisbon, Portugal, July 2007 

■ It’s Not Rocket Science: Building Social Enterprises, Keynote for the 7th Gathering of the 
Social Enterprise Alliance, Atlanta, Georgia, March 2006 

■ Opening Keynote for the Global Social Venture Competition, New York, April, 2006 

■ Keynote for the 7th IAPR Workshop on Document Analysis Systems, Nelson, New 
Zealand, February 2006 

■ Building a Global Library for People with Print Disabilities , a speech for the World Summit 
on the Information Society, Tunis, Tunisia, November 2005 

■ Innovating Information Technologies to Protect Human Rights, a speech for the World 
Affairs Council of Northern California, February 2004 

■ Setting the 2004 Agenda: Technology, speaker at the World Economic Fomm, Davos, 
Switzerland, January 2004 

■ Seizing Market Failure as an Investment Opportunity, Keynote for the Business for Social 
Responsibility Annual Conference, Los Angeles, November 2003. 

■ In the Palm of Your Hand, Keynote for the World Blind Union Asia Pacific conference, 
Singapore, November, 2003 

■ Technology and Human Rights, University of Peradeniya, Sri Lanka, November, 2003 

■ When Markets Fail, Who Responds? Discussion Leader at the World Economic Forum, 
Davos, Switzerland, January 2003 

■ Technology for Nonprofits, with Michael Gilbert, National Gathering for Social 
Entrepreneurs, Minneapolis, MN, December, 2002 

■ Bookshare: Large Scale, Web-Based Accessible Books, TechShare conference organized by 
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the Royal National Institute of the Blind, Birmingham, UK, November 2002 

■ Putting Technology to Work for Development, speech at the United Nations to the joint 
meeting of the World Technology Network and UNOPS, July 2002 

■ Bookshare: The Project for Creating Accessible Books through Computers, at the General 
Session of the National Federation of the Blind 2002 Annual Convention, July 2002 

■ Stanford Social Entrepreneurship Conference, January 2002 

■ The Once and Future Web: Tenth Anniversary of the First U.S. Web Page at the Stanford 
I .inear Accelerator Laboratory, December 2001 

■ Netlmpact Annual Conference at Kenan-Flagler Business School, November 2001 

■ American Council of the Blind Annual Convention, July 2001 

■ Bringing Socially Beneficial Technology into the Service of Humanity. EE380 at Stanford 
University, April 2001 

■ Information Technology in the Service of Human Rights at the Computers, Freedom and 
Privacy Conference, March 2001 

■ Rank Prize Fund Symposium, Grasmere, England 

■ Guest Lecturer for CSUN program in disability leadership 

Professional Associations 

■ Association for Computing Machinery 

■ Institute of Electrical and Electronics Engineers 

■ American Association for the Advancement of Science 

■ Social Enterprise Alliance 

Awards and Public Service 

■ Head of Benetech Delegation, Diplomatic Conference to Conclude a Treaty to Facilitate 
Access to Published Works by Visually Impaired Persons and Persons with Print Disabilities, 
World Intellectual Property Organization, Marrakesh, Morocco (2013) 

■ Member, Global Agenda Council on Measuring Sustainability, World Economic Forum 
(2012-2014) 

■ Member of the Board of Directors, ZeroDivide, foundation investing in community 
enterprises that leverage technology to benefit people in low-income and other underserved 
communities (2007-2013) 

■ Commissioner, Federal Advisory Commission on Accessible Instructional Materials in 
Postsecondary Education for Students with Disabilities, 2010-2011 

■ Duke University, CASE Award for Enterprising Social Innovation, 2011 

■ Brigham Young University, Center for Economic Self-Reliance Social Innovator of the Year, 
2009 

■ AT&T Technology Innovation Award from the Alliance for Technology Access, March 
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2008 

Strache Leadership Award from the California State University, Northridge, 2007 

John D. and Catherine T. MacArthur Foundation Fellowship, 2006 

Technical Advisory Committee Member, National Instructional Materials Accessibility 
Standard, U.S. Department of Education (2005-2008) 

Advisory Committee Member, National Instructional Materials Accessibility Center, U.S. 
Department of Education (2006-present) 

Skoll Award for Social Entrepreneurship . 2004 and 2006 

Fast Company Social Capitalist Award: Top 20 Groups Changing the World, 2004 

Laureate, The 2003 and 2001 Tech Museum Awards 

American Library Association Francis Joseph Campbell Award. 2003 

Schwab Foundation Outstanding Social Entrepreneur of 2003 Award 

Member, the Community Partnership Committee, which oversees a diversity and disability 
agreement with SBC, Inc. 

Runner-up, Yale-Goldman Sachs National Nonprofit Business Plan Competition, 2003 

American Foundation for the Blind Access Award, 2003 

Robert S. Bray Award. The American Council of the Blind 

Winner, Education Category, 2002 Stockholm Challenge 

Fast 50 Champion of Innovation 2002 

Judge, 2002 National Social Venture Competition 

Member, Board of Directors of the Social Enterprise Alliance (2000-2010, chair 2008-2010) 

Member of the Advisory Board, Telecommunications Access Rehabilitation Engineering 
Research Center, a joint effort of the Trace R&D Center of the University of Wisconsin- 
Madison and the Technology Access Program of Gallaudet University, 2001 

Panelist, National Science Foundation Small Business Innovation Research Program, 1998, 
2000, 2003 

Participant, 1998 NSF Workshop for Discussing Research Priorities and Evaluation 
Strategies in Speech Synthesis, August, 1998 

Member, Electronic Information and Technology Access Advisory Committee, a federal 
advisory committee responsible for drafting federal acquisition standards for accessibility 
under Section 508, 1998-1999 

Member, Telecommunications Access Advisory Committee, a federal advisory committee 
responsible for making recommendations to the U.S. Access Board and Federal 
Communications Commission on implementing portions of the 1996 Telecommunications 
Act, 1996-1997 

U.S. Patent Number 5,470,223: System and Method for Tracking a Pedestrian 
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■ Finalist, 1996 Discover Magazine Awards for Technological Innovation 

■ 1996 Access Award, American Foundation for the Blind 

Major Works and Areas of Expertise 

■ Founder and CEO of Benetech . a highly innovative nonprofit company focused on using 
the power of technology to address social needs in areas such as disability, literacy, human 
rights and the environment. 

■ Founder of Arkenstone, Inc., a leading nonprofit organization providing adaptive 
technology for education and employment for people with disabilities and the largest maker 
of reading systems for people with blindness, vision impairment and learning disabilities. 
Developer of the Arkenstone Reader, the first affordable reading system for the blind. 
Designer of Open Book, the first talking Windows program for the blind. Co-inventor of 
Atlas Speaks, the first accessible map software for the blind, and of Strider, a talking GPS 
locator for the blind. 

■ Cofounder of RAF Technology. Inc. , the nation's leading company in optical character 
recognition technology for processing forms in postal and medical applications. RAF's 
software is used to route the United States mail. 

■ Cofounder of Calera Recognition Systems, Inc., the first company to develop omnifont 
optical character recognition that works without user training. 
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Exhibit B 


Documents, Facts, or Data Considered in Forming My Opinions: 

The Public. Resource. Org website, at www.public.resource.org 
The American Printing House for the Blind website, at www.aph.org/ 

The Bookshare website, at www.bookshare.org/cms 
The Learning Ally website, at www.learningally.org 

The National Library Service for the Blind and Physically Handicapped, Library of 

Congress website, at http://www.loc. gov/nls/catalog/?loclr=blognls 

Test Access: Making Tests Accessible for Students with Visual Impairments: A Guide for 

Test Publishers, Test Developers, and State Assessment Personnel, Second Edition, by 

Carol B. Allman, Ph.D., published by the American Printing House for the Blind, and an 

online resource published by the American Foundation of the Blind, available at 

www.aph.org/tests/access2 

Building Assessment Initiatives for Schools: Guidelines to Support the Contract 
Development Process Between Test Publishers and States, available at 
www.afb.org/info/afb-national-education-program/iltli-20Q5-education- 

summary/checklist-for-rfp-building/235 

Window-Eyes screen reader software 

ABBYY FineReader optical character recognition software 

The 1999 Standards in image only PDF format, as provided to me by 

Public. Resource. Org 

The 1999 Standards in TXT format, as provided to me by the Internet Archive 
The deposition of Christopher Butler of the Internet Archive, December 2, 2014 
The deposition of Carl Malamud of Public. Resource. Org, May 12, 2015 
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Exhibit C 
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Test Access 

Making Tests Accessible for 
Students with 
Visual Impairments: 

A Guide for Test Publishers, 

Test Developers, and 
State Assessment Personnel* 
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TEST ACCESS: 

Making Tests Accessible 
for Students With 
Visual Impairments 

A Guide for Test Publishers, 
Test Developers, and State 
Assessment Personnel* 


Fourth Edition 


Carol B. Allman, Ph.D. 


Published by 

American Printing House for the Blind 
Louisville, Kentucky 
July 2009 

AMERICAN PRINTING HOUSE 
FOR THE BUND, INC. 


II 
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*Book Number two in the TEST ACCESS Series, promoting 
accessibility of testing materials for persons who are blind or 
visually impaired 


© 2004 , 2007 , 2008 , 2009 American Printing House for the Blind, 
Inc. With the exception of ETS Guidelines for a Test Reader, which is 
material presented in Appendix G, this document may be copied in 
whole or in part and distributed free of charge for educational and 
nonprofit use as long as appropriate credit is given to the author and 
publisher, and the "Work in Progress" notice is included on each copy. 
No other use of this material is allowed without written permission. 

Work in Progress: This document represents a set of guidelines 
for making tests accessible to students with visual impairments. 
These guidelines are a "work in progress" and will be routinely 
updated and revised as additional information is collected and 
research results are learned. Please address questions, concerns, 
and suggestions regarding these guidelines to the director of APH's 
Accessible Tests Department at 1-800-223-1839 or e-mail them to 
tests@aph.org. 

Disclaimer: Web links in this document were current as of the 
date of publication, but may have become deactivated or modified 
since then. These links are for informational purposes only and do 
not constitute an endorsement or approval of policy, views, 
products, or services of the publishing organization. 

Preferred Citation: 

Allman, C. (2009). Making Tests Accessible for Students with Visual 
Impairments: A Guide for Test Publishers, Test Developers, 
and State Assessment Personnel. (4 th edition.) Louisville, KY: 
American Printing House for the Blind. Available from 
http://www.aph.org. 

Trademarks are of their respective companies. 


American Printing House for the Blind 
1839 Frankfort Avenue 
P.O. Box 6085 
Louisville, KY 40206-0085 
Toll Free: 800-223-1839 
Fax: 502-899-2219 
www.aph.org 
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INTRODUCTION 
Purpose of Document 

The American Printing House for the Blind (APH) is committed to ensuring 
that educational materials are accessible to students with visual 
impairments. Students with visual impairments include those with some 
usable vision, as well as students with no usable vision. This document is 
provided as a guide for making tests accessible in tactile, large print, and 
audio formats. It is anticipated that this guide will be used as a tool for 
implementing appropriate guidelines as test publishers, test developers, test 
editors, and state assessment personnel are developing and adapting tests 
and assessments. Prior planning using the contents of this document will 
help ensure that tests are accessible and will reduce the need to retrofit a 
test. Questions concerning this document, the specific guidelines, or re- 
sources discussed can be addressed to APH's Accessible Tests Department at 
1-800-223-1839 or tests@aph.org. 

Federal and State Mandates 

Federal and many state laws require that all students be assessed through 
state assessment procedures using the appropriate accommodations. In the 
school year 2005-2006, states were required to assess all students' 
progress annually in mathematics and reading in grades 3-8 and once in 
grades 9-12. In addition, by 2007-2008, states were required to assess all 
students' progress in science, at least once in grades 3-5, 6-9, and 10-12. 
Assessment results of all students must be reported publicly, and these 
results are to be incorporated into the state's accountability plan. 

Satisfying these federal requirements necessitates careful attention to 
making tests accessible for all students. Students with visual impairments 
have some unique communication needs that must be addressed as tests 
and assessments are made accessible for them. Converting test items into 
braille, tactile graphics, large print, or audio format fails to guarantee that 
the items are accessible. For example, test items that instruct the student 
to "draw the results of the following" or "write a story based on the 
picture" are not truly accessible to braille readers. Other examples of this 
misconception are discussed within each media-specific section of this 
document. Careful planning during test development can help ensure that 
tests are accessible, while maintaining the rigor intended. 


l 
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Expectations for Students with Visual Impairments 

If students with visual impairments are to participate effectively in state and 
national testing programs, they must have opportunities to learn academic 
skills that will be assessed. These opportunities often are overshadowed by 
special skills training to such students, who may miss all or part of academic 
classes in order to obtain the skills essential for using braille, assistive 
technology, and/or independent living skills, including orientation and 
mobility. 

In addition to providing the training of special skills, school personnel must 
be aware of each student's need for instruction in all academic areas. This 
may entail extended days or school years or supplemental instruction by 
other agencies that serve students with visual impairments. 

Students with visual impairments must spend their educational time 
working toward academic content standards, learning special skills needed 
for independent living, exploring appropriate media for access to printed 
material, and evaluating ways of communication that are effective for them. 
Access to printed material may include braille, tactile graphics, regular print 
with magnification devices, large print, the use of a human reader, auditory 
access, or technology access that provides braille, print, or auditory infor- 
mation. No single method will work for every student, in all situations. 
Educational personnel must ensure that students are exposed to and have 
opportunities to try all options of access. A student's communication mode 
must be based on what works for him or her. See Appendix A for a 
discussion of braille versus auditory access. 

School personnel must maintain high expectations for the education of 
students with visual impairments. If opportunities to learn are present in 
the curricula, students will have the experiences needed to learn difficult 
skills such as map and graph reading, production of graphs and charts, 
reading technical materials, or computation of advanced mathematics. 
Students cannot be denied their right to learn difficult skills just because 
they have a visual impairment. It is these more difficult skills that are being 
assessed routinely on most state and national assessments of student 
progress in reading, mathematics, and science. 


2 
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Universal Design 

In the construction and administration of tests, the process of universal 
design helps to ensure accessibility for a multitude of students. Universal 
design provides the widest range of students the ability to demonstrate 
adequately their skills and knowledge. This process should retain the 
validity of inferences drawn from the test results. 

The concepts of universal design apply to instruction as well as assessment. 
During instruction, universal design enables investigating appropriate 
methods, practicing skills and knowledge using appropriate methods, 
experiencing trial and error to determine the best methods, and 
discovering the success of knowledge and skills learned using the best 
methods for each individual student. During assessment, universal design 
becomes the process of ensuring that the majority of students can 
demonstrate their knowledge and skills. Both aspects of learning, 
instruction and assessment, are driven by the standards of each individual 
state. 

To ensure that an assessment system is fair and accessible to all students, 
states are required to document how they include the principles of universal 
design in the item review process. Generally the principles of universal 
design include (Thompson & Thurlow, 2002, Thompson, Johnstone, & 
Thurlow, 2002) : 

• Attention to an inclusive assessment population, 

• Constructs, including content and cognitive complexity, that are 
precisely defined either through states' standards or the test item 
specifications, 

• Accessible test items, as determined by item writers and review teams 
that include personnel familiar with various media (braille, tactile 
graphics, large print, regular print, and audio), 

• Non-biased test items, as determined by item writers and review 
teams, 

• Test formats, response options, and scoring policies that are amenable 
to various approved accommodations needed by students, 

• Simple, clear, and intuitive instructions and procedures, 

• Comprehensive and relevant language that provide needed distracters 
in test item foils but are not designed to confuse the student, and 

• Maximum legibility of print formats including formats that are free from 
clutter and void of grayscale. 


3 
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Suggestions for applying universal design to item development include: 

• Ensuring that test item writers are trained in concepts of universal 
design, 

• Providing test item writers and reviewers with construct and construct- 
related information during the construction and review of test items, 

• Examining each test item for universal design principles(linguistic 
complexity; cognitive complexity; formatting; bias issues; modalities 
of braille, large print, and audio; and response formats to be allowed), 

• Recommending allowable accommodations for test administration, 

• Re-examining all test items for fidelity to the construct, and 

• Field testing all test items with intended populations. 

The principles of universally designed assessments are the basis for many of the 
guidelines provided in this document. Additionally, the references listed at the 
end of this document have been written by individuals involved in ensuring the 
accessibility of materials for students with visual impairments. This document 
describes guidelines that support braille, tactile graphics, large print, and audio 
production of test items. 


4 
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GENERAL GUIDELINES FOR 
ACCESSIBLE TEST FORMATS 

Students with visual impairments may require testing materials in regular 
print, large print, braille, tactile graphics, audio formats, or some 
combination of these formats. The provision of a test and related materials 
in braille, large print, or audio provided an individual student should be 
based on the medium used by the student, as identified in the 
Individualized Education Program (IEP) document. Alternate format tests 
should be used only by students who use that medium to access printed 
textbooks and other instructional materials. 

Students with visual impairments can be, and must be, made part of the 
state's assessment program through use of accommodations that allow 
them to demonstrate their knowledge and skill acquisition, as outlined in 
each state's standards and assessment system specifications. Regardless of 
the media chosen, students may need access to special materials such as 
braille paper, bold line writing paper, talking calculators, abacuses, raised or 
bold line rulers, braillewriters, slates and styluses, word processors, or other 
materials and devices. A more thorough discussion of accommodations is 
provided in the section on Accommodations in Testing Students with Visual 
Impairments and in Appendix F. 

The following general guidelines are recommended for all formats that are 
developed for accommodating students with visual impairments. Various 
aspects of test construction and implementation are addressed in this 
section. 


Contract Development 

1. Contracts between states and test publishers/producers must include 
provisions for state approved alternate media (braille, large print, 
audio editions of tests, and scripts for oral presentation of tests) 
including answer sheets and practice tests. 

2. Test publishers need to have the capability of providing the test 
administration manual in braille, large print, or audio for test 
administrators who are visually impaired and need accessible media. 
The contract should state if test administration manuals are needed in 
accessible media. 

3. Contracts must include timelines for development, proofreading, 
revising, and production of braille, tactile graphics, large print, and 
audio test formats and accompanying practice materials. 


5 
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4. Contracts regarding accessible media should guarantee that each 
medium of test materials and practice materials is produced by the 
same entity to ensure consistency in format and graphic production 
techniques. Every effort should be made to ensure consistency of 
presentation from one year to the next, and from one level of the 
test to the next. 

5. Contracts may need to include specifications on tools and materials 
that need to be developed or provided to test takers using alternate 
media, e.g. a braille ruler, a tactile or large print protractor, or 
periodic table of the elements, real money for money related test 
items, or some actual objects such as a ball or cube. 

6. Contracts may need to include plans to ship special versions of tests 
separately from regular print versions so that distribution of the 
accessible formats occurs in a timely manner. 

Test Development 

1. Test development must ensure that test score inferences reflect 
intended constructs and not disability characteristics (AERA, 

2000 ). 

2. The construct to be measured must be specified in documents and 
made available to test item writers and reviewers and to accessible 
media producers. 

3. Availability of item specifications is essential in determining 
appropriate accommodation use and in the reproduction of test 
items to be presented in braille, tactile, large print, or audio 
formats. 

4. Test publishers must maintain access to experts in the media of 
braille, tactile graphics, large print, and audio, who can provide 
information concerning test development and transcription and 
tactile graphic design, and who are able to proofread test materials 
before mass duplication, and otherwise ensure that materials are 
provided in a timely and accurate manner. Proofreading the braille, 
large print, or audio version of the test before multiple copies are 
made confirms that the material is readable and that the adapted 
test follows the print copy in numbering and lettering of test items 
and answer choices, and that the graphics are readable and located 
correctly. The proofreader must also check for proper formatting. 

5. Validity issues concerning all accessible formats and accommodation 
needs should be discussed during test development (Phillips, 1994). 
The provision of a test in accessible media should be considered a 
valid accommodation as long as it retains the construct that the test 
was designed to measure. If a performance item requires drawing, 
consider allowing an explanation or description as a valid response 
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option. If such a revision is allowed, scoring criteria must include 
information on this option. 

6. All directions on a test should be worded to allow for alternate 
response methods. For example, use of directions like "circle the 
answer" should be replaced with "indicate or mark the answer." 

7. Specific guidelines on any test format changes, allowable 
accommodations (including time allowances), and general 
assistance that can be provided to the student must be stated in 
the test administration manual or supplemental administration 
materials. 

8. Test item development and review committees should be made aware 
of alternate media issues regarding the use of either complicated or 
nonessential pictures and graphics. 

Item Development and Review 

1. Educators with specialization in the field of visual impairments 
must be included in the test item development process. 

2. All test items must be reviewed by persons familiar with visual 
disability issues to ensure that no test item is biased or 
discriminatory toward persons with visual impairments. 

3. It is recommended that as much information as possible be 
included in the text of a test item. This will help prevent the 
introduction of pictures that contain information necessary for 
selection of the correct answer, but which cannot be adequately 

bra i I led , presented in large print or tactile graphics, or described in 
audio format. 

4. In general, use of "vision specific" language can be maintained, 
e.g., "Look at the following list of animals." 

5. The test item pool must be large enough for bias and item review 
committees to replace items determined to be biased or inaccessible 
in braille, large print or audio formats, or tactile graphics. 

6. A representative sample of students with visual impairments 
must be included in any field-testing of the assessment, as 
prescribed in Standard 10.3 (p. 106) of the Standards for 
Educational and Psychological Testing (1999). 

7. All practice materials must be provided in accessible format at the 
same time that print practice materials are provided. Allow 
sufficient time for accessible format preparation. 

8. Provisions should be made to conduct item analyses for accessible 
format test items. 


7 
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Accessible Test Development 

1. To ensure that quality materials are developed, state assessment 
programs should contract with an agency or persons experienced 

in producing braille, tactile graphics, large print, and audio formats. 

If a multimedia presentation is to be used by test takers, it is 
important that the accessible media producer(s) coordinate pre- 
sentation of the test items between each of the media. 

2. Production of the alternate format test includes the editing, tran- 
scription, reformatting, design, and proofreading of the 
alternate media. 

3. Holding a conference call with all parties involved before the 
accessible media producer begins to review/ edit the test items helps 
to maximize a successful experience and end product. 

4. The name and phone number of the customer's primary contact 
person needs to be provided to the producer of accessible media 
to facilitate timely production. 

5. Accessible format producers will need access to a primary contact 
person, as well as item specifications that include information 
about the skill and construct being assessed. 

6. Test items should be deleted or substituted only if the item cannot 
be provided in braille, tactile graphics, large print, or audio format 
without significantly changing the item and the intent of the 
question. Although not recommended, some test items may need 
to be omitted if they are not adaptable as determined and advised 
by item reviewers with expertise in the format under consideration. 
The deletion or substitution of items should happen infrequently, 
particularly if educators with specialization in visual impairments 
have been involved in the item development process. Attention to 
universal design during test development will also reduce the 
probability that a test item will have to be deleted. 

7. If items are omitted in alternate versions, the test score 

must be rescaled so that braille, large print, and audio format users 
are not unfairly penalized and so that scores can be obtained for 
diagnostic and accountability use. The original numbering system 
should be maintained and the word "omitted" inserted in place of any 
item that had to be omitted. 

8. Responses from the primary contact person regarding questions and 
requests for substitutions require a quick turn-around time in order to 
ensure accuracy and timeliness of delivery of accessible media. 

9. Substituted items should assess the same skill and have equal 
value and validity. Substituted items must maintain the correct 
answer in the same position as that of the original test item. 
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10. All field test items and sample questions must be included in 
accessible format test versions. 

11. Test contracts must indicate preferred publication strategies, such 
as brailling on both sides of the braille paper (referred to as 
interpoint braille), preferred methods of producing tactile graphics, 
and binding of the braille test materials. Assistance with determining 
these specifications is available from APH. 

12. Braille tests are generally produced using contracted braille, the 
typical method for producing braille in which short forms of words are 
used. If the test is for a young child, a new braille reader, or 
someone struggling to learn braille, a test may be needed in 
uncontracted braille, whereby every letter of every word is 
represented by an individual braille cell. 

13. The format of an accessible media test edition must follow the 
print format as much as possible. That is, ideally the number of 
test items and test sections should match that of the print format, 
as should the order of the test items and test sections. Deviations 
from the print version of the test must be outlined in a print copy of 
Test Administration Notes for the altered format. Test Administration 
Notes must include reference to print versions with associated 
accessible format page numbers, identify passages and items by page 
(print and alternate format), and provide indication of any changes 
made to the alternate format. Appendices B and C contain templates 
for creating Test Administration Notes for braille and large print 
formats. 

14. Special requirements, such as an independent proofreading of test 
materials, exact print reproductions of the braille/tactile test items, 
or any print labels to be included on braille or tactile graphics need 
to be considered and included in the contract. 

15. APH's policy in accessible test production includes close collabora- 
tion with, and approval from, test publishers and content 
specialists to ensure that edited items are acceptable as edited. 

16. Test security and confidentiality standards must be upheld during 
the process of developing accessible formats. 

Test Administration 

1. Computers and adaptive technology, electronic note takers, 
cassette player/recorders, the cassettes, CDs, etc., must be 
inspected for proper functioning prior to their use during a test. 

The test administrator or proctor should be instructed on how to 
proceed if equipment fails or malfunctions during administration of 
the test. 
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2. Each test administrator or proctor of a student using an alternate 
medium test or a combination of media should be assigned a 
testing packet that includes a list of materials needed (approved 
technology or other manipulatives, such as a talking calculator, 
braille or large print ruler, braille paper, bold line writing paper, 
raised line graph paper, etc.) 

3. The test administrator or proctor must ensure that special 
tools and materials noted on the student's IEP and used for 
instructional purposes as accommodations are available, as 
needed, to students in the test-taking environment. For example, 
if a visually impaired student routinely uses an abacus in the 
classroom when sighted students are allowed to use a pencil and 
paper for computational purposes, then an abacus must be 
available during a test. See Appendix D for further explanation on 
the use of an abacus in test-taking situations. Specialized tools 
and materials should not be provided if their use presents an 
unfair advantage. 

4. In preparation for test administration, the test administrator needs 
to review the original test(s), the alternate format/s of the test/s, 
the original test administration manual(s), the test administration 
manual/s for accessible media, and the test administration notes 
for the special format/s. These materials should be provided to the 
test administrator under secure and confidential means two full days 
prior to test administration. This time is needed so the test 
administrator can plan appropriately for administration of the test(s) 
in alternate media. 

5. Prior to testing, the test administrator or teacher must ensure 

that the test is available in a student's primary or preferred reading 
medium or combination of media, and that the student has sufficient 
proficiency in use of this medium and related tools such as 
computers, assistive devices, CD players, or braillewriters. 

6. If students are expected to bring select tools and materials to the test 
environment, they need to be notified of this ahead of time. 
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BRAILLE AND TACTILE GRAPHICS 

The information in this section describes methods for developing and 
implementing assessments for students with visual impairments who 
require braille text or tactile graphics. While some technology provides 
auditory access to print, braille is critical to literacy and must be an 
option for those students who routinely use it. See Appendix A for a 
discussion of braille versus auditory access. 

Generally, learning to read braille is no more difficult than learning to 
read print. The tactile process is different from the visual process and 
creates the following considerations: 

• Braille (tactile reading) consumes more time than does visual 
reading, as students who read braille typically read at fewer words 
per minute than do students who read print (Trent & Truan, 1997), 
and 

• Braille reading requires tactile training in page orientation and 
reading and interpretation of tactile graphics. 

Designing Tactile Graphics 

The following are aspects of test items that need special consideration 
when reviewing and designing for production as tactile graphics: 

• Complicated graphics that contain multiple layers or pages of 
information 

• Three-dimensional objects from a particular visual perspective, e.g., 
a top view of a house or pyramid 

• Rotation items that use letters of the alphabet (print letters 
rotated or flipped) 

• Science items that use pictures to demonstrate 
experiments and other scientific concepts or processes (cell, 
digestive or muscle systems, etc.) 

• Map reading items that depend on visually recognizable and 
unlabeled continents, countries, or states, e.g., Africa, Italy, or 
Florida 

• Visual recognition items (interpreting a picture without supporting 
text) 

• Items that require interpretation of complicated drawings, e.g., 
cross-sections of diagrams 

• Optical illusions 
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These types of items frequently require extensive revision during the 
production process. For example, a text-based description in addition to 
the tactile graphic may be needed. The accessible media producer may 
request a test publisher to substitute such items with those that can be 
made more accessible and which will retain similar, if not identical, 
concepts and have the same weighted score. 

Braille Translating (Transcription) Process 

Consideration of the following points will facilitate the production of test 
materials in braille format for students with visual impairments: 

1. Test developers and publishers must ensure that contracts for braille 
materials specify the use of braille transcribers who are certified by 
the National Library Service (NLS), experienced at transcribing tests, 
and knowledgeable of braille formats. Braille formats must be 
modeled after those of the Braille Authority of North America (BANA) 
Guidelines, found in Braille Formats: Principles of Print to Braille 
Transcription , 1997. 

2. As a test is edited for braille transcription, necessary changes will 
be made to make the material accessible to braille readers. 

Correct braille transcription also requires that BANA specifications 

be observed. Simplification and/or labeling of some graphic material 
will likely be necessary. Simplification entails the elimination of some 
artistic features, removal of some superfluous material (without 
eliminating distracters and other text material that is necessary to 
maintain the validity of the test item), or movement on the braille 
page of some text or graphic components for more efficient 
readability by the braille reader (moving a scale, legend, or compass 
rose on a map to a different location). Even simple tactile graphics 
can be very difficult to interpret; some additional labeling may be 
needed for the test taker to read and understand the tactile graphic. 
Note that simplification and labeling are done relative to the 
construct being tested. If during the test editing process, it is not 
clear what is being tested, the test publisher will be consulted for 
clarification. 

3. Reproduced references, such as tables of content, dictionary pages or 
indices, may need to be shortened in the braille test version while 
maintaining correct answer choices and foils. This is done to contain 
the braille item to one page, if possible. 

4. Provision of open-ended items in braille format must indicate to 
the braille reader the amount of space provided for the answer. 
Directions must specify the space provided by suggesting the time 
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needed to complete the item or by indicating the approximate 
page area or the number of lines or paragraphs. Generally, one page 
of print is equal to about two pages of braille unless graphics are 
involved which will add to the page length. Directions may indicate 
that there are four print lines or eight braille lines available for 
responding. 

5. Unnecessary boxes and framing of material may be omitted 
unless the framing provides a separation of graphic material from 
text or encloses a group of scattered or randomly placed objects. 

6. Specific braille codes exist for transcribing literary works, 
mathematics, and science materials into braille. When brailling the 
content of the print version of the test, braille transcribers must 
follow the standards of code for braille transcription. These codes 
are provided in English Braille American Edition , 1994; The Nemeth 
Braille Code for Mathematics and Science Notation , 1972 Revision; 
and Braille Formats: Principles of Print to Braille Transcription , 1997. 
All three manuals are available from the American Printing House for 
the Blind. 

7. An experienced braille proofreader must be utilized for proofreading 
all materials and, in particular, examining all tactile graphics to 
ensure readability and accuracy. 

8. Experienced braille readers might also need to transcribe students' 
braille responses into print for scoring. See the section on Guidelines 
for Braille and Large Print Test Response Transcription. 

9. Braille versions of a test may include transcriber's notes (notes to the 
braille reader from the braille transcriber about the use of special 
symbols, and use of any special formats). Transcriber's notes must 
be written in print within the Test Administration Notes for Braille 
Edition (Appendix B). The number of transcriber's notes in tests 
should be kept to a minimum. 

10. Test security and confidentiality standards must be upheld by braille 
test transcribers, tactile graphic designers, and proofreaders. This 
includes the following: 

• Keeping testing materials in a secure place to inhibit access by 
unauthorized persons, 

• Not sharing information or implying content contained in the 
testing materials with other persons, 

• Maintaining discretion about the work being performed, 

• Returning all materials to the contracting source, and 

• Maintaining confidentiality of test content. 
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Tactile Graphics 

This section offers information regarding the use of tactile graphics when 
testing students with visual impairments. Graphic material, which 
includes maps, charts, graphs, diagrams, and illustrations, frequently 
contains information that is difficult to present in a tactile format. 
Research supports the use of tactile graphics and "the idea that visual 
experience and visual imagery are not required for the perception of 
simple tangible pictures . . ." (Heller, et al., 2002, p. 349). It is possible 
to provide many types of graphic material in braille or raised line 
drawings. However, certain types of graphic materials either cannot be 
provided in braille or tactile formats, or they are so complex that doing 
so produces a graphic that cannot be read and interpreted by the test 
taker. 

Most maps, charts, graphs, and diagrams can be translated into tactile 
form if the test publisher will allow some editing. Editing could involve 
eliminating the shading used solely for visual effect, reducing the number 
of distracters, providing two or three charts to present the same 
information as one complex print chart, using text based descriptions to 
supplement or replace graphics, or using symbols and words with a key 
to provide information. Edits needed to convert print graphics to tactile 
graphics need to be approved by test developers or publishers. 

Most print materials use graphics to emphasize a point, provide another 
format for information, or provide visual appeal. Because graphics are 
common in text, training in reading graphic material and interpreting a 
written description of a graphic are important skills for the student with 
a visual impairment to learn. Guidelines for tactile graphic materials are 
described on the next few pages in terms of general guidelines, design, 
symbols, lead lines, labels, and indicators and scale. 

General Guidelines: 

1. Graphics in mathematics tests must follow provisions of The 
Nemeth Braille Code for Mathematics and Science Notation, 1972 
Revision, BANA (1983). 

2. Decide if a tactile graphic is needed. Omit the graphic if it is 
purely decorative. Consider using a text based description to 
either supplement or to replace all or part of a graphic. 

3. Graphics should be tactually clear and contain only relevant 
information based on an understanding of what is being taught 
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and what the student's task is. Visual information that is 
irrelevant to the meaning or purpose should be omitted. 

4. Graphic material should be simplified without omitting needed 
information or creating an unfair advantage by alluding to the 
answer. 

5. Picture descriptions should be presented concisely within the 
student's test booklet if information in the picture is vital to 
answering one or more test items. 

6. Picture descriptions will appear as needed in transcriber's notes 
in appropriate places throughout the test and must be included in 
the Test Administration Notes. 

7. Some graphics are best handled by supplementing the image 
with a heading, label, description, or key. Edits must be made 
carefully so that the braille reader is not unintentionally given an 
advantage or cue to the correct response. 

8. Consider splitting complex graphics into separate drawings 
showing layers of information, unless this adds complication for 
the test taker. 

9. In general, use texture to add information and draw attention to 
select parts of a tactile graphic. 

10. When necessary, to avoid confusion and accentuate important 
information, use different areal symbols (texture) to differentiate 
between bodies of water and land on maps. 

11. Charts and graphs should be confined to one page when possible. 
If graphics and the accompanying test item require more than 
one page, use facing pages to present graphics and the 
accompanying test item if possible. 

12. If a braille test taker is asked to produce a graphic as part of the 
test item, such a task can be achieved through the use of tactile 
graphic materials that are familiar to the braille reader. Another 
option that may be acceptable to test developers is for the 
student to describe or explain data or other information. This 
option must be approved by the test contractors and included in 
the scoring criteria. The test administrator and the braille 
reader's teacher, using the braille reader's current IEP, must 
collaborate prior to the administration of the test to ensure that 
appropriate materials are provided. For the purpose of scoring, 
student-produced graphics will need to be hand-scored or 
transcribed into a print graphic by persons familiar with braille, 
braille readers, and the content area being tested. See section 
on Guidelines for Braille and Large Print Test Response 
Transcription. 


15 



Case l:14-cv-00857-TSC Document 70-50 Filed 01/22/16 Page 49 of 127 


13. An experienced braille reader must proofread all tactile graphics 
prior to mass production of the braille test to ensure readability 
and accuracy. 

(Kapperman, G., Heinze, T. &Sticken, 1, 2000; Poppe, K. &Otto., F. 2002; Ross, D. 

B. & Robinson, M. C., 2000; Spence, D. & Osterhaus, S., 2000) 

Design: 

1. Avoid clutter and simplify. 

2. "Clutter" occurs when different symbols and lines are so close or so 
similar that they become hard to distinguish. Spacing is the key to 
avoiding clutter. 

3. Symbols and lines closer than W may be difficult to differentiate, 
depending on the medium and tools being used. 

4. Shapes with sides less than Vi" long may not be recognizable. 

5. Use different textures for lines so that test takers know which part of 
a line to follow when two or more line segments cross or meet. 

6. "Simplify" means to eliminate unnecessary elements of the original 
picture. Focus on the relevant parts and omit details that are purely 
decorative or distracting. 

7. When the print picture includes people, animals, objects, etc., replace 
them with simple lines, symbols, and/or labels (e.g., use the label 
"hand" instead of drawing a hand or use a triangle instead of a cat or 
dog). 

Symbols (Lines, Points, and Textures): 

1. Limit the lines, points, and symbols on a drawing to those that can be 
easily differentiated by touch. Use the most prominent symbols for 
the most important features in the graphic. Avoid high or "noisy" 
texture, which draws attention away from the key features. 

2. Be consistent in using symbols within graphics of the same type 
within the same transcription (e.g., always use the same symbol for 
water on maps). 

3. Use different tactile symbols for different types of information (e.g., 
in a map of the United States, the tactile line used to indicate state 
borders should differ from the tactile line used to indicate 
international borders). 

4. Lines, points, and braille must be physically separated by at least 
1 / 8 ". 

a. This distance may need to be extended to at least 'A inch depending 
on the medium and symbols used. 
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b. Apply the 1/8 inch separation rule to all features that are separate, 
even if doing so introduces some spatial distortion. 

Lead Lines: 

1. Use lead lines only as needed; options for lead lines include use of 
keys or notes. 

2. Avoid using arrows as lead lines. 

3. The linear symbol used for lead lines should be different from any 
other lines used in the graphic and should be tactually distinctive but 
less prominent, such as a low relief raised line. 

a. A lead line should begin as close as possible, without causing 
interference, to either the first or the last letter in the label, and it 
should end as close as possible to the feature being labeled. 

b. In general, a lead line should not cross over another line. When this 
is unavoidable, it may make the graphic more readable by breaking 
the lines of the graphic to allow the lead line to "pass through." 

Labels and Keys: 

1. Explain and define all graphic symbols. 

2. Identify all important features (e.g., capitals, bodies of water, etc.) of 
the graphic, even when not labeled in the print version. Place titles 
at the top of the page. Avoid making unlabeled graphics. Exceptions 
may exist in some testing situations. 

3. Position labels closely to the objects to be identified to better ensure 
recognition. Single letters on the graphic should be preceded by 
either the letter sign or the capital sign. 

4. Use two-letter U.S. postal codes where applicable (and other two- 
letter codes where postal codes are not applicable) for labeling state 
names on maps. 

5. Words in labels need not be capitalized if their meaning will not be 
confused and rules of punctuation are not violated. 

6. Place all abbreviations in a key and place the key above the tactile 
graphic. 

7. Present all braille labeling within tactile graphics horizontally. 

Indicator and Scale: 

1. Graphics depicting measurements must maintain accurate and true 
proportions to match the answer choices. If answer choices must be 
changed, the correct response must be located in the same position 
as the original correct response option. 
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2. Position scale and other indicators as consistently as possible, 
preferably above the accompanying tactile graphic. 

3. When it is necessary to change the scale, this fact may need to be 
indicated in a transcriber's note. 


18 



Case l:14-cv-00857-TSC Document 70-50 Filed 01/22/16 Page 52 of 127 


LARGE PRINT TEST FORMATS AND GRAPHICS 

Some students with visual impairments read regular print materials and 
enlarge the print, as needed, by using optical devices. Some read large 
print materials. This section offers information regarding the develop- 
ment and implementation of assessments for students with visual 
impairments who require large print materials. Generally, two popular 
methods exist for enlarging tests. The regular print test can be enlarged 
through photocopying, or an electronic version of the test can be 
manipulated to reformat test items and enhance the readability of text 
and graphic as needed. The latter method is preferable unless issues 
outlined in this section have been addressed during the test development 
and the regular print test has been designed using universal design 
principles. Manipulating an electronic version of the test can best yield a 
large print version that incorporates the optimum reading mode for the 
student who uses large print. 

Generally, reading skills that are difficult for a person with low vision who 
reads print include the following: 

• Reading at a speed commensurate with regular print readers, 

• Reading for extended periods of time, 

• Visual scanning and skimming of text, 

• Shifting gaze from a picture or graph to test item and back 
again, 

• Shifting gaze from test booklet to answer sheet documents, 

• Visually capturing an entire picture, 

• Moving from one line of text to the next, 

• Locating pictures and text presented in random locations on 
page, 

• Interpreting pictures (particularly complex pictures), 

• Differentiating between subtle colors and patterns used in 
pictures or graphs, and 

• Filling in answer choices on regular print answer documents. 

Consideration of these points, particularly in relation to universal design 
of test format and printed text, will facilitate the production of test 
materials in large print format. As well, most of these guidelines are 
applicable to regular print tests that may be used by students with low 
vision. Information provided on font, spacing, shading and contrast, 
pagination, and test booklets is a summary of work done by Elaine 
Kitchel, presented as "Reading, Typography, and Low Vision," a PowerPoint 
presentation (APH, 2002). Research completed by G. E. Legge et al., 
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(Reported in "Psychophysics of Reading" 1985 through 2002 in Vision 

Research) supports the guidelines listed in the following section. 

Test Format 

1. Large print versions of a test and test practice materials should be 
reformatted from the regular print version so that adaptations can be 
made to font style, print size (point size), line length spacing, 
shading, graphics, and the number of items on a page. 

2. Text should consistently begin at the top left-hand side of a page. 
Titles of pictures or graphs should appear at the top of the graphic. 

3. Labeling should be presented horizontally rather than vertically as a 
general rule. Exceptions may be labeling of y-axes on graphs, etc. 

4. Items that typically present the most difficulty during conversion to 
large print format include the following: 

• Complicated, multi-shaded drawings with extensive details, 

• Grayscale drawings that provide little contrast, 

• Colors that cannot be differentiated by persons with color 
blindness, 

• Large maps that cannot be contained on one page if enlarged, 

• Extensive charts with multiple columns, and 

• Charts and graphs that extend over several pages. 

5. If testing materials are enlarged merely through photocopying 
(not recommended), the font size will vary depending on the original 
print font. When tests are enlarged, the font size of all text, 
including labels on graphs, rarely meets the 18-point size required. 
Enlarged materials must be reviewed and proofread before mass 
copying or distribution to ensure that print and background contrast 
are adequate, that pictures and graphs are readable and complete on 
the page, and that items assessing measurement are accurate and 
have viable answer choices. 


Fonts 

1. Print measuring 18 points or larger is considered large print. Point 
sizes between 12 and 16 points are considered enlarged print. 

2. Occasionally a test in a print size larger than 18 point will be 
requested. In such cases, the publisher must determine if material 
can be adequately presented in a larger point size. 

3. Decisions about the size of print and font style must be made by 
the test publisher and discussed with a person who has knowledge 
of large print use and the intended test takers. 
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4. Font styles that are decorative or cursive should not be used. 
Standard sans serif fonts with easily recognizable characters are 
recommended. Verdana, APHont, Arial, Antique Olive, and Helvetica 
are reliable choices for readability. Note: APHont, a font for people 
with low vision, developed by the American Printing House for the 
Blind (APH), embodies characteristics needed by low vision readers as 
identified by research. A free version of APHont is available from APH 
at http ://www.a ph.org/products/aphont qet.html . 

5. Large print should have x-heights (distance from the top to 
bottom of a lower case "x") and t-heights (distance from the bottom 
of the "t" to the cross bar of the "t") of at least 1/8" with a 
thickness of 2 points. Eighteen point Verdana, APHont, Antique 
Olive and Helvetica meet this standard. 

6. The use of bold print, underlined print, or quotation marks for 
highlighting text is preferable to using italics. Italics should only be 
used when absolutely necessary. Sample test items, if provided, 
should be presented in the same font size and style as that used for 
the actual test items. Letters incorporated into math problems, e.g., 
letters within algebraic equations, are also more readable when 
displayed in a non-italic, sans serif font. 

7. Headings and subheadings (captions, titles of diagrams and charts) 
should be larger and bolder than other print and set in a font style 
that differs from that of the general text. Acceptable typefaces for 
this use include Arial Black, Helvetica Black Bold, Lucida Sans Bold, 

Era Bold ITC, Verdana Bold, Antique Olive Bold, and Helvetica Bold. 

8. All text, including labels and captions on graphs, pictures, diagrams, 
maps, charts, equations, exponential numbers and letters, subscripts 
and superscripts, notes, and footnotes, must be presented in at least 
18-point type, in order to meet the APH definition of large print 
(Kitchel, 2001). 


Spacing 

1. Leading or spacing between lines should be at least IV4 spaces to 
allow persons with low vision to effectively move from line to line 
in the text. 

2. Block style formatting and 1" margins are recommended. 

3. Format should include justification of left margins, and unjustified 
right margins (rag right) for ease in reading and transferring from 
line to line. Avoid the indentation of paragraphs. 

4. Avoid dividing words between lines. 

5. Columns of text, excluding graphic material, should be at least 39 
characters in line length. Generally, for efficient reading, columns 
should be avoided. 
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6. Test items and accompanying diagrams, pictures, and graphics 
should be located close to each other and on the same page if 
spacing permits. If this is impossible, test questions, diagrams, and 
answer choices should be placed on facing pages or follow closely so 
that page turning is reduced to a minimum. 

7. Research indicates that readers with low vision and readers with 
normal vision read a wide-bodied font faster and with better 
comprehension than they read a variable-spaced font (Mansfield, & 
Legge, & Bane 1996). 


Shading and Contrast 

1. Grayscale and shading should be avoided, particularly when 
information needed for answering a test item is provided. 

2. The highest possible contrast should be used for text and back- 
ground, with attention to the use of color. Certain color combi- 
nations other than black and white may be unreadable to 
persons with low vision or persons with color blindness. A good 
rule of thumb on use of colors is to use colors that are far apart on 
the color wheel and avoid using colors that have similar saturation 
(color depth). Blue and yellow, for example, provide a high degree 
of contrast when used together. Red and green should be avoided 
because they are the most troublesome colors for persons with color 
blindness. 

3. Large print must not be used over a background design or other 
graphic material. 

4. Glossy paper may cause unnecessary glare. Dull finish paper in white, 
ivory, cream, or yellow is recommended and best complemented 
with black print. 

5. Unnecessary boxes and framing of material should be omitted unless 
the framing provides a separation of graphic material from text or 
encloses a group of scattered items. 


Cautions for Use of Recycled Paper 

Whether recycled paper is appropriate or not for use by individuals 
with low vision depends on its color and its thickness. The color 
cannot tend toward gray, blue, or green. If it is slightly gray, blue, or 
green (and many recycled papers are) it can substantially reduce 
contrast. What seems like a minor contrast difference to a sighted 
person can be a big contrast difference to a person with low vision. 
However, if the tint of the paper tends toward beige, peach, pink, or 
yellow then it would be fine. In addition, there should be no speckles 
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in the paper. Best color choices for recycled paper would be cream, 
beige, or white. Finally, the paper should be thick enough to prevent 
bleed-through of inks. The paper needs to be thick enough to allow 
printing on both sides of the sheet with no bleed-through. The same 
cautions apply to materials for persons with color blindness or color 
vision deficiencies (Kitchel, 2009) (see section on color vision issues 
below). 


Pagination 

1. Repagination of original test materials is preferable to increasing the 
overall page size. 

2. While double-sided pages are generally preferable, avoid double 
sided copying if print will "bleed" or show through or otherwise 
obstruct clear reading. 

3. Where blank pages must appear, type the words "Blank Page" 
near the top left hand side of the page. 

Format of Test Booklets 

1. Depending on test length, large print copies may need to be 
separated into several booklets. 

2. Generally, the test booklet should be no larger than 9" x 12", 
particularly for young students as well as other students with various 
physical conditions. 

3. The binding of the large print booklet(s) should allow each page to 
lie completely flat for whole page viewing and ease of handling. 

Large Print Graphics 

The following guidelines provide information concerning the use of 
graphics in testing students with visual impairments who use large print 
formats. Work by the Large Print Atlas Focus Group (2001), who met at 
the American Printing Flouse for the Blind, is included in this discussion. 

The complexity of some graphic materials prohibits their being provided 
in large print unless they are modified to become more readable when 
enlarged. Most maps, charts, graphs, and diagrams can be enlarged if 
the test publisher agrees to some editing. Editing could involve the 
elimination of shading, the reduction of some distracters, the insertion of 
a key, or the separation of one chart into two or three. 

Guidelines for large print graphics include the following: 
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1. Graphics in large print must exhibit good contrast, clarity, and 
accurate details and information. 

2. No test item should rely solely on a picture for information 
needed to answer the test item. In consideration of universal 
design, include a text description of every non-text item. 

3. Generally, pictures should be retained in the large print format. 
Editing for shaded material and clarity may be necessary. Some 
pictures that would need extensive editing and provide little or no 
cues for the large print reader may be considered for elimination. 
Purely decorative graphics should be deleted. 

4. Overlaid print on a diagram or graph should be avoided. While 
visually pleasing in some instances, this technique is difficult for 
persons with low vision to read. 

5. Multi-color graphs that use closely related colors may conceal 
vital information from the test taker who is unable to 
distinguish between the colors. Two to three contrasting colors 
or black and white are recommended. 

6. All graphs should contain short, descriptive headings or titles. 

7. Compass points, numbers, and vital information on graphs must 
be enlarged sufficiently for the low vision reader. 

8. Map symbols must be easily distinguishable and relevant. 

9. Map legends should appear near the top left hand corner of a 
map, if possible, and include a visually distinctive border. Use 
contrasting colors and distinguishable symbols rather than 
reproducing different sizes of the same symbol. 

10. If possible, map scales, too, should be positioned near the top 
left-hand corner of the map. 

11. Labels should be arranged within the boundaries of the country or 
state borders whenever possible. 

12. Symbols used should be reasonable and meaningful 
representations, e.g., a fish for fishing. 

13. Boundaries between countries should be bolder and thicker than 
boundaries between states or provinces on a map. 

14. Pictures and graphs used in test questions requiring measurement 
must be true to the size intended in order to ensure that a 
correct answer is available. 

15. Test publishers and contractors will need to address the degree of 
accuracy that is expected for questions involving measuring or 
drawing. For example, some large print readers may not be able 
to distinguish between 7/16" and 8/16" on a ruler. If at all 
possible, specially designed measurement devices, such as large 
print rulers and protractors, should be provided for students in 
both the classroom and testing situations. 
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16. If a graph or table does not exceed one page in the original 

materials, then the large print version should be edited to fit on 
one page, if possible. Pertinent information and distracters must 
be maintained. 
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USES OF COLOR FOR SIGNAGE, GRAPHICS, 
TEXT, TESTS AND POWERPOINT 
PRESENTATIONS TO BE VIEWED BY PERSONS 
WHO ARE COLOR BLIND OR COLOR 
VISION DEFICIENT 

Introduction 

Color is critical to the conveyance of meaning in signage, graphics, text, 
PowerPoint® presentations, tests, and other written presentations. 

However, some people, specifically those with color discrimination difficulties, 
need special consideration when color planning for educational purposes. 

Virtually all color-deficient individuals have varieties of red or green 
deficiency. (Blue deficiency is rare indeed, with only about .001% of the 
population having it.) Color blindness is normally diagnosed through clinical 
testing by a licensed practitioner. 

PROBLEMATIC AREAS 

When one considers educational materials for students who are color blind or 
color deficient, some problematic areas come to mind: 

• Use of gray-on-gray bubble sheets on test answer sheets 
(scannable answer documents) 

• Maps with indistinguishable adjacent colors, such as coloring Spain 
brown and Portugal green 

• Graphs with indistinguishable adjacent colors 

• Use of text over graphic backgrounds, as when a poem or other 
text is superimposed over a photo or drawing 

• Test questions which depend upon color identification for correct 
answers 

PREPARATION OF MATERIALS FOR PERSONS 
WHO HAVE COLOR VISION DEFICIENCIES 

Color is one of the most important aspects of visual communication and can 
be employed to generate interest or to communicate ideas or feelings. Yet 
colors for an audience with members who have color discrimination problems 
should be selected carefully to avoid conveyance of unintended meaning. 

This is especially true in educational and testing materials. Many of these 
materials rely on good color perception for the interpretation of graphs, 
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charts and illustrations. Yet even the most carefully thought-out graphic may 
lead the user to an incorrect answer because of poor color selection. 

• Select colors carefully. Besides black and white, most color blind 
individuals can only see two colors, blue and caramel (golden brown). 
Red, yellow, orange, and green take on shades of caramel; purple takes 
on shades of blue when viewed by a person with colorblindness. 

• Less is more. Too many colors used thoughtlessly can confuse and 
negate the message of a graphic. Settle on four or fewer colors and stick 
with them. Black and white are counted as colors when designing 
graphics, even though they are not usually considered colors when 
talking about vision. 

• Use contrasting colors. Contrast is an important influence on the 
legibility of graphics, especially for persons with color discrimination 
problems. Substantial contrast, i.e., the use of dark values with light 
values, between the color of the foreground and the background should 
be employed. High contrast makes materials easier to read by both 
persons with colorblindness and those with typical vision. Light letters on 
a dark background or dark letters on a light background are most 
legible, but remember the actual colors of those combinations are 
important. 


CONTRASTING COLORS APPROPRIATE FOR PERSONS WITH COLOR 

PERCEPTION DIFFICULTIES 
(in order of best contrast value) 

• Use black and white. 

• Use dark blue and white. 

• Use black and bright yellow. 

• Use dark blue and bright yellow. 

• Use dark brown and white. 

• Use pale blue and black. 

• Use yellow and purple. 

Notice that yellow is recommended as a common color for graphics to be 
used by persons with poor color discrimination. This is because yellow 
maintains luminance longer than any other color. Even though it is perceived 
as a light caramel color by persons with color blindness, it holds its 
brightness longer than any other hue, and therefore maintains its contrast 
when paired with a dark color. 
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COLOR COMBINATIONS TO BE AVOIDED 




Avoid gray with any color, even another value of gray. 

Avoid red with any color except white or blue. 

Avoid green with any color except white. 

Avoid brown with any color except white or blue. 

Avoid purple with any color except yellow or white. 

Avoid orange with any color except blue or white. 

Avoid two values of the same color, such as light blue and dark 
blue. 

Avoid a neutral color with any other neutral color. 


The importance of proper attention to color selection cannot be overlooked 
when developing tests for individuals or groups that have color vision or 
color perception deficiencies. 
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GUIDELINES FOR AUDIO 
VERSIONS OF TESTS 

This section is written to provide assistance in the development and 
implementation of accessible tests for students with visual impairments 
who require audio versions of a test. Audio formats include cassette tape, 
video, CD, computer-based, or spoken (read aloud) test versions. When 
an audio version of a test is administered, the audio version should be 
accompanied by a print, large print, or braille version of the test, or a 
large print or tactile graphic supplement at the very least. In this multi- 
media approach, a student can access illustrations or other visual 
material that may not be described, or only minimally described, on the 
audio version of the test. 

Some illustrations can be described orally in an accurate manner, while 
other graphic material cannot be described without revealing the answer 
or providing an unfair advantage to the audio user. A complete script for 
audio versions should be written with the assistance of a content expert 
and provided to test administrators. 

Audio versions of a test serve to standardize oral delivery of the test 
content and may reduce the number of school staff needed for proctoring 
or administering tests orally. Consideration of these points will facilitate 
the production and administration of test materials in audio format. (See 
Appendix A for a discussion regarding braille versus auditory access.) 

Production of Audio Tests 

1. Test publishers may only have the capability of providing one version 
of a test in audio format. The version selected should be parallel in 
content and difficulty to other versions. 

2. An experienced test editor should be involved in editing for an audio 
presentation of a test. The audio edition will need to be coordinated 
with other media in which the test will be provided. 

3. Audio versions should be developed using the resource The Art and 
Science of Audiotape Book Production published by the National 
Library Service for the Blind and Physically Handicapped, Library of 
Congress. Requirements for narrator, monitor, and proofreader are 
provided in this document. 

4. The National Braille Association in Tape Recording Manual, Third 
Edition (1979) provides instructions for reading mathematics 
instructional materials. This source recommends that graphic 
materials be described, if possible, and accompanied by print or 
tactile versions of the graphics. Such modifications need to be 
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approved by the test publisher. Moreover, the audio descriptions, 
print, and tactile versions of the material need to be coordinated. If 
different departments within the same company or different vendors 
are responsible for developing and producing the accessible media, 
one source should be responsible for ensuring that the media are 
coordinated to the extent specified. 

5. Narration of print materials must follow National Library Service 
(NLS) specifications of minimum acceptable requirements 
(Specifications #300 and #304). 

6. Test publishers must ensure that narrators follow confidentiality and 
security assurance standards of test materials. Security measures 
taken when working with audio formats should mirror those required 
for handling print or braille materials (Kentucky Core Content Test 
Administration Manual Supplement ; 2000). 

7. Test publishers must give attention to packaging and labeling of the 
audio test. Audio tests may be packaged for each individual student, 
with the appropriate print and/or braille supplements needed by the 
student and the test administrator. 

8. Ascertaining whether the audio format will be administered on an 
individual basis or in a small group setting is important, as the 
information on the audio format and the information to be provided 
by the test administrator may vary depending on the setting. 

9. The audio test should instruct the student to stop at certain 
points. Audio procedures must ensure that test takers work only on 
allowable sections of the test. For example, selected subtests may be 
recorded on separate cassette tapes or CDs and then collected as 
required. 

10. Directions for navigating through the audio version should be 
provided in print for the test administrator. 

11. Test publishers must select an experienced narrator with appropriate 
voice, speech, accuracy, and pronunciation skills. (Pronunciation 
resources are available from NLS.) 

12. Narration must be evaluated and proofread to ensure that test 
content is conveyed accurately and that questions are presented 
without unintended emphasis on correct answers. 

13. A person with identical test materials should monitor the narrator to 
ensure accuracy during audio production. A third person should be 
used for the proofreading of audio materials. 
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Administration of Test Items in Audio Formats 

1. Students using audio versions of a test should have had an adequate 
amount of experience using the specific audio medium and audio 
equipment independently before the testing situation. 

2. Test administration materials should indicate the equipment 
required by the student for using audio versions of a test. For a 
cassette tape version of a test, a standard two-track tape player/ 
recorder and headphones will be needed. An audio test on CD will 
require use of a common CD player or a digital talking book player, 
depending on the audio file type of the CD. Regardless of the player 
used, a backup player capable of playing the same audio medium 
should be available. Access to electrical power or sufficient batteries 
for player/recorder use should be indicated. Test administrators 
should be instructed to inspect the equipment functions before 
testing begins. 

3. Test administrators will need to monitor student "movement" 
through audio versions to ensure that the student maintains the 
appropriate place in the test and to ensure that the audio version is 
playing properly. When using a two-sided cassette tape, students 
may need to be reminded to play the other side of a tape. Prior to 
administering the test, and in the absence of students, test proctors 
should spot check audio formats to ensure proper operation of the 
audio medium and equipment. 

4. Students using an audio version of a test must be seated in a quiet 
area and away from other students so that other students are not 
disturbed by the audio medium or equipment operation. Students 
can choose to use headphones. 

5. Provisions must be made in the test administration manual for the 
malfunctioning of audio equipment. Students may have to be tested 
at a later time if malfunctioning occurs. Students must not be denied 
access to the administration of a test because of equipment 
malfunctioning or failure. 
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GUIDELINES FOR ORAL READING OR 
SIGNING OF A TEST 

Students who are visually impaired or deafblind may need the accom- 
modation of a reader or sign language interpreter. Occasionally, an audio 
version of a test is not produced, and a test publisher, developer, or 
assessment personnel will allow the reading or signing (use of sign 
language) of a test or portions of a test for students whose Individual- 
ized Education Program (IEP) specifies this accommodation. Before using 
oral reading or sign language as an accommodation, careful attention 
must be given to the constructs being measured. For example, if a 
section of the test is designed to assess reading as a decoding skill, then 
the reading or signing of the test to a student would invalidate the results 
for the intended purpose. In these instances, consider an alternate test 
or redefine the construct for the individual student. Always check with 
the test publisher or test developer to determine the construct intent 
and accommodation use for particular sections of a test. 

State policy dictates if passages and stimuli can be read aloud and/or 
signed for large scale statewide assessments. Check with the District Test 
Coordinator or with the State Department of Education assessment office 
for the policy in your state. 

For the oral reading or signing accommodation to be allowed on statewide 
assessments, a student must have had exposure to and have used this 
accommodation during daily instruction and on classroom tests. This is 
especially true when mathematical symbols and technical or content- 
related language is being read and accessed. It is recommended that a 
student have access to print or braille graphic material even if the reading 
or signing accommodation is used. 

The Educational Testing Service recently posted on their web site ETS 
Guidelines for a Test Reader (July, 2000), which have been made available 
in Appendix G of this document through special permission from ETS. This 
document is helpful in outlining the characteristics of a good reader, 
providing general information for readers, indicating special considera- 
tions for multiple-choice tests, addressing mathematics reading, and 
providing test center procedures for using a reader. In addition, 
consideration of the following points will ensure appropriate provision of 
oral reading or signing of a test or portion of a test: 

1. Test security and confidentiality standards must be upheld. 
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2. The test purpose must be specified to ensure that reading or signing 
a test or portions of a test do not invalidate results or preclude how 
the results will be reported. 

3. An experienced test editor and professionals involved in working with 
students who require readers or interpreters need to be included in 
the team of persons that adapt tests which are to be read or signed. 

4. A prepared script must be provided for test administrators to ensure a 
consistent, standardized presentation of the test items. 

5. A reader or sign language interpreter must have skills in 
presenting various types of test materials. For example, someone 
familiar with mathematical symbols is needed in order to correctly 
read and convey higher level math formulas and equations. 

6. A standard video presentation of the test in sign language is 
recommended to ensure quality, consistency, pacing, and accuracy. 

7. The person selected to read a test to a student should have the 
characteristics of good voice quality and appropriate speed and tone. 

8. The person signing a test must be a trained interpreter and be able 
to translate in the same method of sign language typically used by 
the student. It is not recommended that the student's teacher be 
the interpreter for the testing situation unless a second person is 
present to monitor for quality and fairness during administration of 
the test. 

9. Voice inflection (regional dialect and pronunciation) familiarity is 
recommended. 

10. The narrator or interpreter must avoid voice inflection that stresses 
or otherwise indicates the correct answer. 

11. The interpreter must avoid facial expressions and body language that 
may cue the correct response. 

12. Students tested through oral reading of the exam must be tested 
individually to prevent the testing situation from becoming a group 
effort. Moreover, testing individually helps ensure that each student 
receives the specific oral reading structure required by his or her 
individual needs. 

13. Directions can be read or signed to groups of students. 

14. The interpreter or reader must be allowed to review test 
administration materials and items on the test to ensure that they 
have knowledge of the vocabulary/signs required for that 
assessment. This is important so that the reader/interpreter does 
not accidentally cue the correct response. The reader/interpreter 
should have access to pronunciation dictionaries, sign language 
dictionaries and technical skills manuals to use as references. It is 
important that the reader/interpreter sign a confidentiality agree- 
ment before reviewing the materials to ensure test security. 
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15. Oral readers and interpreters will need to pause at appropriate 
intervals to provide the student an opportunity to answer test items 
or access graphic material provided in print or tactile formats. 

16. Graphic materials may be described as detailed in the prescribed 
script, but must also be made available in print or tactile formats. 

17. Oral readers or interpreters must avoid providing an answer to a 
student's question concerning clarification of testing content. Doing 
so would provide an unfair advantage. Developing some standard 
responses to students' questions prior to the testing situation is 
helpful. For example, you can encourage the student to listen to or 
watch the signing of the question again. 

18. Readers or interpreters may need to provide multiple readings or 
signings of passages, parts of passages, or items. Unless instructed 
otherwise in the Test Administration Manual, professional judgment 
and any guidance provided in the IEP should be used to determine 
the number of readings necessary. 

19. If the oral reader or interpreter is also completing an answer sheet 
for a student, the transfer of answers must be performed carefully to 
ensure that the student's answers are recorded as intended. See 
section on Guidelines for Braille and Large Print Test Response 
Transcription. 

20. Two readers or interpreters should be used for presenting a test or 
portions of a test to a student. Using two readers or interpreters 
helps ensure accuracy of test presentation and provides the 
opportunity for readers or interpreters to rest after 15-20 minutes 
of presenting test material. 
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ACCOMMODATIONS IN TESTING STUDENTS 
WITH VISUAL IMPAIRMENTS 

The use of accommodations during testing is intended to level the 
playing field for any student with a disability. There are, by nature of 
the disability, certain accommodations that are needed by students 
with visual impairments. Not all of them discussed in this section are 
intended for use by all students with visual impairments. Likewise, 
some needed by students with visual impairments may not be 
presented here. 

Accommodations and various technologies exist to provide learners 
with visual impairments access to academic instruction and tests. The 
term "technology" comes under the definition of assistive technology 
as described in federal law and is considered an accommodation to the 
testing of students with visual impairments. 

The need for one or more accommodations is the decision of the 
Individualized Education Program (IEP) team and must be recorded on 
a particular student's IEP. Accommodations used during testing should 
generally match those used by the student for classroom instruction, 
assuming they are familiar and effective for the student. Their use is 
determined by evaluating factors unique to each student and must be 
implemented as outlined on the IEP. Evaluation of their effectiveness 
for an individual student is highly recommended. Further, students 
must be trained to use accommodations. For example, providing a 
test orally by a qualified person or on computer might actually penalize 
a student who has not been trained to listen to orally presented 
material or trained to use a computer for assessment. 

Accommodations should be periodically evaluated to ensure that they 
are still effective for the student. Some may need to be eliminated or 
revised when and if the student arrives at a point where he or she 
either does not need the accommodation, it is ineffective, or it is not 
the most effective option available. If an accommodation is needed by 
a student and is not on the list of those approved for state use, the 
local test administrator should contact the state assessment office to 
request a review of its use. 

The next segment presents general as well as specific accommodations 
for test takers with visual impairments who use braille, large print, 
and/or audio formats. See Appendix F for additional information on 
this topic. 
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Types of Accommodations for Students 
with Visual Impairments 

Presentation Accommodations 

1. Braille, large print, and audio are accommodations that some 
students with visual impairments will use interchangeably. A student 
may, for example, read a passage in braille and prefer to access a 
table or chart in an enlarged version of the test. Therefore, students 
should be allowed to use a large print (or regular print with 
magnification) and a braille version of the test, if requested. 

2. Some students who are visually impaired may need to have read to 
them the test directions or some of the test items, as long as those 
items read do not assess reading as a decoding skill. See section on 
"Some Guidelines for Oral Reading and Signing of a Test. " 

3. Computer-administered testing is an accommodation that has 
received some attention through research, though studies 
concerning its benefit are inconclusive (Tindal & Fuchs, 1999). 
Generally, however, when a student uses a computer for daily 
classroom activities, then this accommodation may prove useful 
during testing if the concepts being tested are not undermined. There 
are several programs and peripheral materials that can be used to 
adapt the computer for use by persons with visual impairments. 
Screen readers, text to speech technology, and keyboard access 
through braille or switches are available. Depending on the construct 
being tested, test administrators must verify that the student is 
inhibited from accessing software or hardware that may provide an 
unfair advantage. For example, if a student's basic math skills are 
being assessed and the intent is not to use a calculator, then the 
keyboard functions or software used for computations must be 
blocked. 

4. When testing allows the use of non-scientific or scientific calculators, 
students with visual impairments should be permitted to use an 
equivalent device that has been adapted for use by a visually 
impaired user. Should a state provide non-scientific or scientific 
calculators for the sighted population taking the test, then non- 
scientific or scientific, talking calculators should be provided to 
students with visual impairments who are taking the test. 

5. An abacus is often useful for students when mathematics problems 
are to be worked without a calculator. The abacus functions as paper 
and pencil for some students with visual impairments who have 
received instruction and practice on the use of the abacus. See 
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Appendix D for the position paper "Use of an Abacus in Test-Taking 
Situations." 

6. Students may want to use manipulative devices, such as a ruler or 
template, to maintain placement on a line of braille or print. Other 
tools available for use by visually impaired students include braille or 
large print rulers and protractors, raised line or bold line graph 
paper, or raised line or bold line writing paper, to name a few. 

Contact the American Printing House for the Blind (APH) toll free 
at 1-800-223-1839 to request a catalog of available accessible 
materials, or visit APH on the Internet at www.aph.org. 

Response Accommodations 

1. Students with visual impairments may need to present answers 
orally to a test administrator who completes the answer sheet. See 
section on Guidelines for Braille and Large Print Test Response 
Transcription. 

2. Students with visual impairments may need to write answers in the 
test booklet or on separate paper using a braillewriter or slate and 
stylus. 

The student's answers must then be transcribed and transferred to 
the answer sheet. See section on Guidelines for Braille and Large Print 
Test Response Transcription. 

3. Students may need to write answers using a word processing 
program, to be transferred to the answer sheet. Depending on the 
construct being tested, test administrators must verify that students 
are inhibited from accessing software or hardware that may provide 
an unfair advantage. For example, if a student is responding to a 
writing prompt and the writing will be judged based on correct 
spelling and grammar, then the spell check function and grammar 
functions must be disabled. 

4. If a student must draw or somehow demonstrate a response, then 
accessible tools and materials that are typically used by the student 
for instructional purposes must be made available in the testing 
environment as long as no unfair advantage is provided. For this 
type of open response item, it is very important that scoring criteria 
be well defined and allow for variation in response methods. 

Timing Accommodations 

1. The use of extended time for test completion is a testing 

accommodation that has received considerable attention since state 
testing and accountability systems have been implemented. Research 
investigating the use of extended time has yielded little conclusive 
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information about its benefit (Tindal & Fuchs, 1999). However, 
students with visual impairments will usually require extended time 
during testing because using braille, print, and audio formats require 
more time than does reading print with acceptable visual acuity. A 
study by Wetzel and Knowlton (2000) suggests that experienced adult 
braille readers may need no more than 50% more time than the 
stated duration, with additional time allowed for the manipulation of 
an audio device or the marking of an answer sheet. In contrast, an 
earlier researcher found that braille readers with far less braille 
reading experience than the subjects mentioned in the Wetzel and 
Knowlton study may need between 2 and 3 times as much time as 
their sighted peers to read the same material (Nolan, 1966, p.l). 
Traditionally, extended time fortesting readers who are visually 
impaired has been IV 2 times, and for braille readers time allotted has 
been 2 times the amount allowed for regular print readers 
(Lowenfeld, Abel, & Hatlen, 1969, pp. 91-92). Regardless of the time 
allowed, the student should be carefully monitored to ensure that time 
is being used appropriately. If students need an inordinate amount of 
time, educators may need to investigate the efficiency of the chosen 
reading mode or initiate remediation to improve speed. Generally, 
timing accommodations should be individualized according to the test 
taker's reading rate and testing situation (Wetzel & Knowlton, 2000). 
See Appendix E on the "Use of Extended Time." 

2. Reading braille, print, or listening to material presented orally, 
especially when accompanied by graphic material, can be a fatiguing 
and often frustrating experience in a high stakes testing environment 
for students with visual impairments. Therefore, students may need 
several brief sessions in which to take the test. Additional break 
options should also be considered. 

3. Students may need to be tested over a longer time period, a week 
rather than two days, for example. However, any alteration of the 
timetable will necessitate close supervision to ensure test security. 

4. Students may need to be tested at different times of the day 
depending on their optimal functioning time. 

Setting Accommodations 

1. Some students with visual impairments may need to be administered 
a test or select subtests individually, or in small groups as 
recommended on their IEP, to ensure that the test accommodations 
needed by the students are implemented without interfering with 
the concentration and test taking results of other students. 

2. If a student is recording answers by using technology that is noisy or 
is recording answers orally, then he or she must take the test 
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individually and under the supervision of a test administrator in order 
to avoid distracting or influencing the responses of other students. 


Specific Accommodations in Testing Readers Who 
Require Enlarged or Large Print 

Enlarged print is that which is 14 point, 16 point, or regular print that 
has been enlarged using magnification devices. Large print is 18-point 
type and larger. Enlarged print and large print are accommodations. 

Some students may choose to use a regular print test and enlarge it 
manually with a magnification device with which they are familiar. 
Magnification devices include eyeglass-mounted magnifiers, free standing 
or handheld magnifiers, and electronic equipment such as the closed 
circuit television (CCTV) or a computer that has text enlargement 
software installed. These devices do not provide a student with an unfair 
advantage. Rather, they are devices that the student requires to access 
print, and they should be allowed as standard accommodations. 

Proper lighting and freedom from glare, while sometimes overlooked, are 
critical for many readers with visual impairments. Lighting that has been 
adjusted to suit the student's particular visual needs and minimize glare 
will help promote sustained reading efficiency. 

Specific Accommodations for Audio and Oral Test 

Administration 

Students using an audio version of a test or having the test orally 
administered as an accommodation should also be allowed to have print 
(large print or regular print with a magnification device) and braille 
versions of the test, if requested. A student may wish to listen to a 
passage by way of audio, but access a table or chart in a large print or 
braille version of the test. Listening to an oral description of a geometric 
figure can be difficult or impossible to follow unless an enlarged graphic 
or a tactile graphic accompanies the oral description. 
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GUIDELINES FOR BRAILLE AND 
LARGE PRINT TEST RESPONSE 
TRANSCRIPTION 

Some students with visual impairments will use the accommodation of 
oral response, written response (on the test booklet or on paper other 
than the test answer sheet provided by the test publisher), or taped 
response. Each of these accommodations requires that a person 
transcribe the answers onto the answer sheet or booklet that will be 
scored. These guidelines are provided to ensure that transcription is 
performed appropriately. 

1. Confidentiality of the test materials and the student's individual 
responses is critical. Transcribers must treat the testing materials 
and the student responses in a secure and confidential manner to 
ensure test and student identification security. 

2. Response transcribers must know braille if transcribing braille 
responses. 

3. It is best if the response transcriber is a "neutral" person, not 
someone with a vested interest in the student's scores. 

4. Response transcribers must provide the exact answers that the 
student has written using the same punctuation, spelling, and 
grammar structure. They cannot guess what the student might 
have meant if answers are incomplete. 

5. It is recommended that the response transcriber have a second 
person proofread the responses to ensure accuracy and fairness to 
the student. When transcribing graphics that a student has 
produced, two transcribers should work together in transferring 
student answers to the answer sheet or booklet. 

6. For a period of time, student responses must be maintained in a 
secure file with test name, copyright year, form and level 
administered so that the student's actual responses can be 
reviewed if questions arise. 
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REPORTING TEST RESULTS OF STUDENTS 
WITH VISUAL IMPAIRMENTS 

Following the requirements of federal law, the scores of students who 
take assessments in accessible format must be reported for account- 
ability purposes. When reporting the results of students with visual 
impairments, care must be taken to protect the student's privacy while 
appropriately representing the test score in consideration of the 
accommodation(s) used. Students must not be penalized for use of 
approved accommodations that do not change the test construct and do 
not provide an unfair advantage to the test taker. Reporting of scores 
should be a consideration during the test development phase so that all 
parties understand the purpose of the testing and how the results will be 
reported and used. 

Reporting Test Results for Braille Editions 

For most assessments, braille test versions should be regarded as 
appropriate accommodations for students who use braille daily. Any 
rescaling of braille test versions that is performed because of item 
omission should be reported. The scores of those students taking a test in 
braille should be considered valid as long as the test has been prepared 
using the guidelines presented in this document. Students who read 
braille daily need to use braille to respond to test items. This dual use 
provides an instructional/ assessment validity match. Extensive efforts 
to "prove" a braille test invalid because of a difference in format are 
neither recommended nor useful. If the purpose of a test is to determine 
educational skill progress, the validity can be addressed by confirming 
that the media used for instruction matches that which is used for 
assessment. 

Reporting Test Results for Large Print Editions 

Large print versions of tests also qualify as appropriate accommodations 
for use during the assessment of students who use large print daily. 

Unless the assessment has been reformatted, the large print version is a 
camera-enlarged version of the original version. If the test is altered 
through removal of shading, or other clutter from graphics, the use of 
the large print format should be considered an appropriate and valid 
accommodation. Generally, if reformatting is performed in a manner 
preserving the original test content, the reformatted version should be 
considered valid. Producers of large print must work with test publishers 
to verify that the test material has not been altered in content or 
purpose in order to maintain test validity. 
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Reporting Test Results for Audio and 
Orally Administered Tests 

Regarding most assessments, the use of audio and orally administered 
tests should be considered appropriate accommodations for students who 
use audio and oral formats on a routine basis to access materials. For 
tests that assess reading as a decoding skill (visually or tactually), audio 
and orally administered versions may change the skill being tested, and 
this should be noted in any report of scoring. 
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ALTERNATE ASSESSMENTS 

The guidelines presented in this section address some specific issues 
related to accessibility of alternate assessment for students who are 
blind or visually impaired. The guidelines addressing general state 
testing that are presented throughout this book are appropriate 
considerations for providing alternate assessment materials for some 
students in this population; however, the specific needs of this group 
must be discussed because so many are non-readers. Best practices in 
this arena are still being formulated. 


General Issues 

Students who meet the criteria for alternate assessment, by definition 
of the federal law, are those students who have significant cognitive 
disabilities (often referred to as the 1% population assessment). 

As allowed by federal law, some states have chosen to provide a 
second alternate assessment for those students who are not expected 
to meet the state standards as demonstrated on the general state 
assessment within the same time frame as students taking the general 
state assessment. In addition, these students are to be working 
toward the state standards using modified achievement standards as 
identified by each state. In some states, this alternate assessment 
(generally referred to as the 2% population assessment) mirrors the 
general state assessment with the exceptions of having fewer answer 
choices and in some cases using simpler language in the test items. 

Since alternate assessments are very similar to the general state 
assessment in most cases, the same requirements for accessibility are 
applicable for the alternate assessment as are outlined for the general 
assessment. 

The needs of students who are blind or visually impaired and have 
additional disabilities that may qualify them for these alternate 
assessments, must be considered in the planning and developing of 
alternate assessment formats and items. Providing accessibility for this 
population of students requires that test publishers and state 
personnel have access to professionals who are familiar with braille, 
large print, and regular print and know the learning styles of these 
students. 

Because many students who take alternate assessment have limited 
reading ability, it is expected that students who are blind who qualify 


43 



Case l:14-cv-00857-TSC Document 70-50 Filed 01/22/16 Page 77 of 127 


for alternate assessment will have very limited braille reading 
capabilities as well. Likewise, students with low vision who qualify for 
alternate assessment may have limited ability to read print or large 
print. 

While providing a general assessment in braille for accessibility 
purposes has its challenges, the provision of an alternate assessment 
in tactile format can be even more challenging. Alternate assessments 
typically have formats that either require the student to answer 
questions by looking at a visual stimulus or demonstrating skills from a 
checklist of desired tasks. Alternate assessments often include 
performance tasks, such as picture identification or demonstration of 
skills using manipulatives. Generally, demonstration of specific skills 
on alternate assessments can be easily accommodated to allow the 
student who is blind or visually impaired to perform tasks in the usual 
way they perform tasks in the classroom. Validity and reliability can 
usually be maintained when accommodations have been well- 
documented on a student's IEP and assessment report. 

Considerations in Alternate 
Assessment Design 

• Because reading is an issue for the population of students taking 
alternate assessment, test items often require picture identification. 
Test administrators must be able to describe the pictures for 
students who are blind. All test items need to be reviewed to assure 
that the picture can either be described (accessible) without giving 
the answer away or that the picture is not needed (inaccessible) 
and has been omitted. Keeping the task appropriate to the student 
who takes the test is crucial: For example, it is appropriate to ask a 
student about the function of an object (i.e. Which of these can you 
eat— a book, a rock or a banana?). An item would not be accessible 
if it asked: "Which of these pictures shows a banana?" In this later 
example, naming the pictures (book, rock, banana) would give the 
answer away. 

• It is recommended that the test publisher provide picture descrip- 
tions for the test administrator. Picture descriptions should be 
developed in conjunction with content experts and state 
assessment personnel, keeping in mind the construct (skill) being 
assessed and the cognitive level of the students taking the test. 

• If pictures cannot be described without compromising the test item, 
it is preferable to present manipulatives (objects) to students in lieu 
of pictures. If this is allowed by the state, such objects should be 
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real objects (i.e. actual familiar classroom objects and shapes such 
as pencils; paper; books; notebooks; toys; food; geometric shapes; 
and counters) and not replicas. 

• Replicas or miniatures of animals or of other large objects are not 
appropriate for use because they cannot be distinguished by the 
student. These should be used with caution and only if the student 
is familiar with the models or miniatures. 

• Real money should be used rather than a tactile representation. 

• Tactile representation of simple graphs (charts) is appropriate. If 
the test administrator is allowed to read the chart or graph, a script 
for reading it is preferred and should be included in the test 
administration manual. 

• Tactile representation of shapes (circles, squares, stars, rectangles, 
etc.) is appropriate and should be used in place of letters, animals, 
and people to present counting or other mathematics items. 

• If the skill being assessed is not reading, then test administrators 
should be allowed to read aloud all words and passages used in the 
test items. Care must be taken not to give vocal clues by 
emphasizing certain words. 

• If the construct being assessed is decoding of words (i.e. reading) 
and simple words and/or passages are provided, then these words 
and passages must be provided in braille. It is important for state 
assessment personnel to determine if contracted or uncontracted 
braille, or both will be provided. This can best be done by surveying 
a sample of teachers who work with this population to determine 
which format is preferred or by requesting school districts to specify 
the number of alternate assessments that are needed in contracted 
braille and the number needed in uncontracted braille. Making this 
decision part of the ordering process ensures that the appropriate 
braille test format is provided for each individual student. 

• Large print is defined by research as optimal at 18 points. A sans 
serif font should be used for best readability. 

• It is important that any pictures or graphics provided in print or 
large print are clear, uncluttered, black line drawings with no grey 
scale. 
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APPENDIX A 

BRAILLE VERSUS AUDITORY ACCESS: A 

DISCUSSION 

Federal law requires that consideration be given to accommodations in 
testing students with disabilities. With this focus comes the responsibility 
of the educator to identify needed and useful accommodations for 
students with disabilities. For students with visual impairments, 
accommodations that provide access to print can vary considerably. The 
range of accommodations includes braille, tactile graphics, large print, 
regular print with magnification, auditory media, or any combination of 
these accessible media. This discussion suggests methods for identifying 
the most appropriate accessible media, identifies uses of braille and audio 
materials, and provides recommendations for consideration in choosing 
testing media. 

Since the early 1990s authors have identified methods of evaluating the 
"mode of reading" or method of print access for students with visual 
impairments (Koenig & Holbrook, 1993; Wormsley & D'Andrea, 1997). 
Federal law indirectly requires that print access be evaluated by defining 
the consideration of braille as a mode of reading for students with visual 
impairments as part of the Individualized Education Program (IEP) 
process. A major part of the early and ongoing assessment of a visually 
impaired student's unique needs is the use of various media to access 
printed materials. Identification and use of appropriate media includes: 

• Determination of the student's primary and secondary sensory 
channels for learning through observation of the student's use of 
vision, use of touch, and use of hearing in familiar and unfamiliar 
settings, at structured times and unstructured times, and in outdoor 
settings as well as indoor settings (Koenig & Holbrook, 1993) 

• Attention to the student's current print access needs, instruction and 
remediation in accessible media or alternate media, and recognition 
of future needs in print access for the student 

• Provision of initial sensory channel identification and ongoing sensory 
channel use to determine changes in use and need for instruction in 
additional media access skills 

• Instruction in a variety of accessible media that could be used by the 
student 

• The opportunity to learn skills that enable the student to choose the 
appropriate medium for various tasks 
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Specifically, the appropriate uses for braille are determined by each 
individual who uses braille. Most blind individuals access printed materials 
by using a combination of media. One issue in using braille has typically 
been the lack of braille materials. Currently, there are improved methods 
of providing braille materials through the expansion of technology. 
Computer software and hardware that translate print to braille, provide 
braille displays, and emboss braille through a translation program are 
used to provide most braille text in a timely fashion. It should be noted 
that print with highly graphic and technical content does not translate to 
braille easily and with the type of accuracy expected for testing 
materials. 

Congress has recently passed legislation that will ensure accessibility of 
instructional materials in braille for students with visual impairments. 
While assessment materials are not included in this legislation, it seems 
that making instructional materials readily available and accessible will 
drive the need for a similar pattern in the testing arena. 

Persons with visual impairments routinely use auditory means to access 
large volumes of literary or recreational reading material, such as novels 
or magazines. The expansion of technology and the ability to translate 
printed text into speech has enabled persons with visual impairments to 
access information via computer software and/or hardware. Additionally, 
many persons with a visual impairment make use of a screen reader for 
print access, a skill that requires some training. 

The availability of a wide range of ways to access print is important for 
persons with visual impairments. This range of availability should exist 
for students but should not be confused with, or used as a replacement 
for, the skill of learning to read (decode language). If society values the 
reading of materials as a decoding skill, then access to printed material 
for students who are visually impaired must include the learning of 
reading through tactual or visual processes. For some individuals the 
reading process is too tedious to be efficient. These individuals may 
choose to use primarily auditory materials as adults, but as students 
they should be given the opportunity to learn reading as a decoding skill. 

The skills involved in reading braille, reading print, and listening to audio 
materials are unique to each medium. Therefore, during the development 
of test items, test publishers must be clear about which constructs are to 
be assessed by a particular item. If reading as a decoding skill is to be 
assessed, then a fair assessment can only result if the student is 
provided with material that can be visually or tactually read. If 
comprehension is the construct being assessed, then the test developer 
must determine whether reading comprehension or listening 
comprehension is the skill to be assessed. Comprehension would need to 
be defined to ensure that students are using appropriate accommodations 
when taking a particular test. 
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The following recommendations should be reviewed when considering the 

use of braille or audio materials for students with visual impairments: 

1. Braille and tactile graphics interpretation should be taught as media 
access skills so that students may learn reading as a decoding skill 
and have the option of using braille and tactile materials. 

2. Auditory listening skills should be taught as a media access skill so 
that students can learn listening comprehension skills and have the 
option of using audio materials. 

3. Test publishers must be certain about the construct being assessed 
on all test items. This enables educators and test administrators to 
make valid judgments about appropriate accommodations for 
students with visual impairments during test administration and helps 
to ensure correct interpretations of test results. 
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APPENDIX B 

TEMPLATE FOR TEST ADMINISTRATION 
NOTES FOR BRAILLE TESTS 

Name of Test: 


Edition of Test: 


Section: 


Preliminary Pages Transcriber's Notes: 


Special Symbols Page: 


General Test Direction Notes: 


Print Page 
Number(s) 

Braille Page 
Number(s) 

Accompanying 

Test 

Administration 
Manual Page 
Number(s) 

Item 

Number(s) 

Notes 
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Explanation of Fields on 
Test Administration Notes for Braille Tests 


Name of Test: Provide the full and exact name of the test. 

Edition of Test: Provide the copyright or other edition listing to further 
identify the test. 

Section: Provide the section name and other identifying information. 

Preliminary Pages Transcriber's Notes: Provide in print the exact 
wording of transcriber's notes that refer to preliminary pages in the 
braille version of the test. Indicate the page number of the transcriber's 
notes. 

Special Symbols Page: Provide in print the exact wording of the special 
symbols page that may be present within the braille version of the test. 
Indicate the page number of the special symbols page. 

General Test Direction Notes: Provide information about the methods 
a student may use when responding to test items that differ from print 
test versions and which require special equipment or attention. 

Print Page Number(s): Provide the location of test material within the 
regular print version of the test. 

Braille Page Number(s): Provide the location of test material within 
the braille version of the test. 

Accompanying Test Administration Manual Page Number(s): 

Provide the page number(s) in the test administration manual that 
correspond with each regular print test page. 

Item Number(s): Provide the test item number(s) that appear on that 
print page. 

Notes: Provide comments that indicate transcriber's notes specific to 
particular pages, changes made to the braille version of the test, and 
changes made to directions, as listed in the test administration manual or 
on the test. 
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APPENDIX C 

TEMPLATE FOR TEST ADMINISTRATION 
NOTES FOR LARGE PRINT TESTS 

Name of Test: 


Edition of Test: 


Section: 


General Test Direction Notes: 


Print 

Page 

number(s) 

Large Print 

Page Number(s) 

Accompanying 

Test 

Administration 

Manual 

Page Number(s) 

Item 

Number(s) 

NOTES 
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Explanation of Fields on 

Test Administration Notes for Large Print Tests 


Name of Test: Provide the full and exact name of the test. 

Edition of Test: Provide the copyright or other edition listing to further 
identify the test. 

Section: Provide the section name and other identifying information. 

General Test Direction Notes: Provide information about the methods 
a student may use when responding to test items that differ from print 
test versions and which require special equipment or attention. 

Print Page Number(s): Provide the location of test material within the 
regular print version of the test. 

Large Print Page Number(s): Provide the location of test materials 
within the large print version of the test. 

Accompanying Test Administration Manual Page Number(s): 

Provide the page number(s) in the test administration manual that 
correspond with each regular print test page. 

Item Number(s): Provide the test item number(s) that appear on that 
print page. 

Notes: Provide comments that indicate changes made to the large print 
version of the test and changes made to directions as listed in the test 
administration manual or on the test. 
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APPENDIX D 
POSITION PAPER: 

USE OF AN ABACUS 
IN TEST-TAKING SITUATIONS 

By Terrie Terlau and Fred Gissoni 

Definition and Description 

The mathematical abacus is a frame with beads or balls that can be slid on 
wires or in slots for calculating or teaching arithmetic (The American 
Heritage Dictionary of the English Language, 1996). The abacus has been 
used as a calculation device in Europe, Japan, China, and the Middle East 
since the third century A.D. It continues to be used widely in Japan 
( http://www.svuzan.net/enqlish/education/education.htmn . 

The Cranmer abacus was developed as a calculation device for persons who 
are blind or visually impaired and is currently produced by the American 
Printing House for the Blind (APH: Abacuses, 2001). The Cranmer abacus 
frame is made of high impact plastic, measures 6-1/8 x 3-1/4 x 7/16 inches, 
and contains thirteen vertical rods and one horizontal cross bar. Four beads 
can be moved vertically on each of the thirteen rods below the cross bar and 
one bead can be moved vertically along the rods above the cross bar. 

Abacus Functionality 

When calculating with the Cranmer abacus, vertical rods represent units, 
tens, hundreds, etc. Numbers are recorded and manipulated by moving 
beads toward the cross bar on their respective rods. 

The abacus is a passive device. It is not a calculator or a slide rule. The 
abacus does not perform mathematical operations. It does not contain 
information that would enable an abacus user to achieve calculation results 
without a solid knowledge of mathematical concepts and relationships. 
Abacus users produce calculations as a result of their understanding of the 
behavior of numbers, not because of any inherent property of the abacus. 

Both abacus and pencil-and-paper users must learn strategies for performing 
mathematical operations. The primary difference in the activity of abacus 
and pencil-and-paper users is that pencil-and-paper users apply and record 
steps in these operations by writing while abacus users apply and record 
these processes by moving abacus beads. 
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Persons who are blind or visually impaired and who have had appropriate 
abacus instruction can use the abacus to perform addition, subtraction, 
multiplication, division, and square and cube roots. The abacus does not 
permit permanent storage of problem solutions because beads must be 
rearranged to perform subsequent problems. After each calculation using an 
abacus, answers can be recorded in a variety of formats including braille, 
large print, voice recording, word processing, or dictation into an electronic 
device. 


Position Statement 

Whenever a test-taker is allowed to use a pencil and paper for working 

calculations, an abacus should be considered an equivalent substitution. 
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APPENDIX E 
POSITION PAPER: 

USE OF EXTENDED TIME 

Introduction 

In addition to the use of braille and large print, the use of extended time 
is also a commonly used accommodation for students with visual 
impairments. This position paper provides a brief summary of the results 
of research on the use of extended time in testing students, while 
suggesting best practices for implementing this accommodation. 

Research 

For several years, researchers have suggested that students with a visual 
impairment need more time to complete assignments and tests (Harley & 
Lawrence, 1984; Kederis, Nolan, & Morris, 1967; Morris, 1974; Spungin, 
2002; Bradley-Johnson, 1994). 

Moreover, some researchers have reported results indicating that 
students with a visual impairment generally read at a slower rate than 
students without a visual impairment (Packer, 1989; Legge, et. al., 
1985, 1989; Wetzel & Knowlton, 2000). Not only does the reading of 
braille and large print generally require more time than reading regular 
print, but the time needed to explore and interpret pictorial information 
presented as tactile or enlarged graphics can be a tedious and time- 
consuming process. Therefore, extended time seems to be an obvious 
accommodation for this population. Some suggested time extensions 
based on classroom experience or research include 

• 1.5 to 2 times for students with low vision (Gompel, van Bon, & 
Schreuder, 2004), 

• 2.5 times for braille and 1.5 times for large print (Morris, 1974), 

• 1.5 times for all students with a visual impairment (Spungin, 2002), 

• 2 times for braille (Kederis, Nolan & Morris, 1967), 

• More than 2 times for braille and a little less than 2 times for visually 
impaired readers who read print (Packer, 1989), and 

• .5 times for experienced adult braille readers (Wetzel & Knowlton, 
2000 ). 

The most recent synopsis of research on accommodations demonstrates 
the wide range of results among studies seeking to validate the use of 
extended time during testing. Based on the varied results, authors 


63 



Case l:14-cv-00857-TSC Document 70-50 Filed 01/22/16 Page 97 of 127 


recommend that a well-designed test for standard administration be 
untimed (Tindal & Haladyna, 2002). 

Research conducted by the National Center on Educational Outcomes 
(NCEO) summarizes at least four studies in which the use of extended 
time had a positive effect on student test scores. NCEO provided 
preliminary results of a Universal Design Research project which suggest 
that unlimited time reportedly helps students "think better," a conclusion 
drawn after interviewing students who had completed a universally 
designed test (with no time limits) and a regular test (with time limits) 
(Presentation: Universal Design Research, C. Johnstone & A. Morse, June 
24, 2003 at CCSSO Large Scale Assessment Conference, San Antonio, TX). 

Several authors seem to agree that timed conditions may not allow 
students to reflect their full abilities on achievement tests (Tindal & 
Fuchs, 1999) and that adequate time should be provided for all students. 
Parr, et. al. (1996) argue that extended time examinations taken under 
ideal circumstances can be more equitable and practical than timed 
examinations. In another investigation, Marquart (2000) found that 
extended time failed to significantly improve the test scores of disabled 
students. The author, however, does conclude that extended time likely 
produces a more accurate measure of a student's skill by helping to 
reduce test anxiety and by allowing a greater opportunity to use good 
test taking strategies. 


Conclusions 

Extended time is a commonly used accommodation for students with 
visual impairments. Some literature concerning the subject recommends 
that the accommodation of extended time be of specific duration, e.g., 
2.5 times for braille readers and 1.5 times for large print readers. 
Certainly, a topic in need of additional information is a comparison of 
time used among the following: a braille reader who must explore and 
interpret tactile graphics, a large print reader who must visually examine 
and synthesize enlarged graphics, and a sighted student using regular 
print test materials. Moreover, several current researchers suggest 
placing less emphasis on designating a uniform, "one size fits all" duration 
of extended time as an accommodation for disabled students during 
testing. Rather, these researchers suggest that the accommodation of 
extended time consist of "adequate time." That is, a specific length of 
time, which must be determined by educators through careful 
assessment of the student's physical disability, skills, and needs. In lieu 
of extended time, some test administrators are finding that more 
frequent breaks are effective for braille and large print test takers. Once 
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the need for, and duration of, adequate time and/or breaks has been 
assessed, educators should include that information on the student's IEP, 
ensure use of the accommodation, and monitor its use. 

Position Statement 

To implement extended time or adequate time for students with visual 
impairments, four basic steps should be followed: 

1. Assess the need for extended time and frequent breaks. 

2. Include specific information about extended time and the need 

for breaks on the student's Individualized Education Program (IEP). 

3. Ensure that extended time and frequent break accommodations 
are implemented as specified during testing. 

4. Monitor the student's use of extended time to assure that the 
student uses extended time/break time appropriately and that 
the student is on task. 
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APPENDIX F 
POSITION PAPER: 

ACCOMMODATIONS FORTESTING STUDENTS 
WITH VISUAL IMPAIRMENTS 


By Carol Allman, Ph.D. 

Introduction 

Accommodations and technologies exist for the purpose of providing a 
disabled student with access to academic materials that may otherwise be 
inaccessible. The term "technology" comes under the definition of assistive 
technology as described in federal law and is considered an accommodation. 
Accommodations and assistive technologies needed by students with visual 
impairments should be outlined on the student's Individualized Education 
Program (IEP). These accommodations should be monitored periodically for 
their effectiveness with the individual student and revised or updated as 
appropriate. Any accommodations provided for students during the 
testing window should be ones typically used by that student in the 
classroom and not new or unfamiliar ones. 

This paper provides an overview of accommodations in testing that might be 
effective for students with visual impairments and should be documented on 
their IEP. Five major categories of accommodations that include 
presentation, response, setting, scheduling, and special tools are discussed. 
Not all of the accommodations presented in this paper are intended for use 
by every student with a visual impairment. Likewise, some accommodations 
needed by students with visual impairments may not be discussed. 

Determining Accommodations 

The need for accommodations is the decision of the Individualized Education 
Program (IEP) team and must be recorded on the IEP. Accommodations used 
in testing should match those used by the student for classroom instruction. 
Accommodation use is determined by evaluating factors unique to each 
student and must be implemented as outlined on the IEP. Evaluation of the 
effectiveness of accommodations for individual students is highly 
recommended. Further, students must be trained to use accommodations. 
For example, providing a test orally or on a computer might actually penalize 
a student who has not been trained to listen to material presented orally or 
trained to use a computer for assessment. Accommodations should be 
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continually evaluated to ensure that they are effective for the student. Some 
accommodations should be eliminated if the student arrives at a point where 
he or she either does not need the accommodation or the accommodation is 
ineffective. 


Presentation Accommodations 

Students with visual impairments have several options for accessing test 
materials. According to data collected by the American Printing House for the 
Blind (2003), 9% of the visually impaired student population use braille as 
their primary mode of reading. Approximately 26% use large print materials, 
while only 6% are auditory readers who would require test materials to be 
presented in audio format. Prereaders (27%) may use auditory materials 
until they learn braille or print. Of the nonreaders (32%), some may use 
braille, large print, and audio on a very limited basis. However, there are 
many whose significant cognitive disability would inhibit them from 
successfully using braille, large print, and audio materials. Most of these 
students are involved in educational programs that do not rely heavily on 
traditional reading media and modes of learning and communication. This 
population of students may use augmentative or tactile communication 
systems and might qualify for alternate assessment in the statewide 
assessment program. The remainder of the visually impaired school-aged 
population who are readers access standard print materials with or without 
low vision aids. 

Braille, large print, magnified print, and audio presentation are 
accommodations that allow visually impaired students access to the testing 
environment. Some of these students may use a combination of these media 
to complete a single test. A student may, for example, read a passage in 
braille and prefer to access a table or chart in a large print or magnified 
format. Students using an audio version of a test as an accommodation 
would also be allowed to use print (large print or standard print with a 
magnification device) and/or braille versions of the test, if requested. 

Further, a student may prefer to listen to an orally presented passage but 
access a table or chart in a large print or braille version of the test. If a 
multimedia presentation is used, the various media must be coordinated to 
ensure accuracy and accessibility. It should be noted that computer-assisted 
testing is becoming very popular and requires special attention to be 
accessible for students with visual impairments. 
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Braille and Tactile Graphics 

Braille is a system of raised dots that represent words and letters. It is used 
as a presentation method for those students who typically read braille for 
classroom instruction. Braille may be presented as contracted (using short 
forms for words as outlined in English Braille Code) or in uncontracted format 
(using no short forms, i.e., spelling each word letter by letter). Most students 
will use the standard contracted braille. A few students, such as those who 
are just learning braille in the early grades or who are newly blinded, may 
need uncontracted braille to access a test. 

The production of a braille test is a unique process that often necessitates 
the review and limited editing of test directions and test items so that the 
items are understandable when presented in braille and tactile graphics 
format. Such editing may involve subtle word changes to directions 
(replacing "circle the answer" with "mark the answer"), relocation of stimulus 
information (moving the question above a graph or chart), simplification of a 
graph or chart (removing extraneous information without deleting answers or 
foils), or replacing an item that cannot be reflected in braille with an item of 
equal weight, value, and difficulty (replacing an item that requires strictly 
visual skills, such as visual illusion, with a similar item that assesses the 
same concept and is more accessible to blind students). 

However, an item need not be replaced or omitted simply because it is 
presented in a manner that requires some visual interpretation. For example, 
the concept of understanding a shadow and what causes a shadow is an 
important concept for a blind student to understand. Therefore, this skill can 
be assessed through use of descriptions and tactile graphics. If, however, a 
test or particular subject includes a high percentage of visual items, then 
consideration may be given to substituting some of the "visual" items. 
Students who read using braille are expected to meet the same standards 
that other students meet, even though they are doing so tactually. The 
process of editing a test for braille production should in no way simplify or 
reduce the difficulty of the test material. 

Once test material has been edited for braille transcription, qualified persons 
will transcribe the print into braille by using the recommended edits and 
guidelines for braille transcription and formatting. The transcribed braille test 
must be proofread and produced so that the braille reader receives a high 
quality test in the same timely manner as sighted students receive their test. 
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Large Print Text and Graphics 

Large print is considered such when it is 18-point type and larger. Enlarged 
print is typically that which is 14 point, 16 point, or standard-sized print that 
has been enlarged using magnification devices. Enlarged print and large print 
are accommodations. 

Large print should be produced by using an electronic version of the test to 
reformat the test so that fonts are larger, fewer items are on a page, 
graphics are contained on one page, answer choices are presented with the 
questions, and attention is given to improving the contrast and reducing the 
shading and gray scale that interferes with reading the material presented. 
The process of using a photocopier to enlarge test content should be avoided 
since this method lacks the control needed to ensure that all test material 
(exponential numbers, footnotes, and graphic material) is represented in a 
readable point size, that text is clear and without gray scale interference, 
and that problems dealing with measurement are presented accurately. For 
example, a butterfly measuring two inches in the standard print test must 
remain two inches in the large print version. 

Some students will use magnification devices (discussed in more detail in the 
section of this paper on special tools considerations) with large print or with 
standard print to access test materials. 

Therefore, it is important that the standard print version of a test exhibit 
good contrast and a clear print style to allow effective use of magnification. 

Audio 

Generally, students with visual impairments should be expected to read 
materials by using print or braille. Access to print is a critical literacy skill for 
all individuals. However, where audio presentation is allowed, and for 
reducing the time needed to complete a test, some students who are visually 
impaired may need directions or some test items presented orally to them. 

Audio presentation of print materials is a presentation accommodation 
allowing for all or part of a test to be presented on cassette tape, CD, 
computer and specialized screen reader or text reader software, or read 
aloud to a student. Students should use these accommodations only if they 
use audio media for classroom instruction. The skill of listening to spoken 
material and manipulating a computer, cassette tape player or CD player is 
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different from the skill needed to read and interpret print or braille. 

Therefore, navigating through a cassette tape, computer with screen reader, 
or audio CD in a testing environment requires practice. Further, the test 
purpose must be specified to ensure that oral presentation of a test or 
portions of a test do not invalidate results or preclude the reporting of test 
results. For example, if the reading skill of decoding print (or braille) is being 
assessed, audio presentation of the text could invalidate the purpose of the 
test. 

The transfer of test material onto audio tape requires a process similar to the 
construction of test materials in braille. Print text must be edited for audio 
presentation, produced in audio format by experienced audio engineers, and 
then proofed for accuracy. Additionally, any graphic material must be 
described and provided as a supplement in braille, large print, or standard 
print. Accurately describing graphic material requires attention to the critical 
components of the graphic and careful consideration of which details can be 
included in or omitted from the description without providing the answer or 
excluding the foils imbedded in the question. 

If a test or part of a test is to be read to a student, there are recommended 
practices for ensuring that this accommodation is provided correctly: 

• A reader must be skilled in presenting various types of test materials. 

For example, a reader familiar with mathematical symbols is required for 
the correct delivery of higher level math formulas and equations. 

• The person selected to read a test to a student should exhibit good voice 
quality, appropriate regional dialect, pronunciation, speed, and tone. 

. The reader must avoid voice inflection that stresses or otherwise 
indicates the correct answer to test items. 

• Prior to the testing situation, difficult words within the test material must 
be reviewed by the person assigned to read the test. Pronunciation 
dictionaries should be used as references. 

. Readers must pause at appropriate intervals so that the student has an 
opportunity to answer test items or access graphic material provided in 
print or tactile formats. 

• Readers must avoid answering a student's question concerning 
clarification of testing content. Doing so would provide an unfair 
advantage. Developing some standard responses to students' questions 
prior to the testing situation is helpful. For example, instead of 
answering a student's question about test content, the reader can 
encourage the student to listen to the question again. 

. Readers may find it necessary to provide multiple readings of passages, 
parts of passages, or items. While addressing the needs and requests of 
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the test taker, the reader should also use professional judgment to 
determine the number of readings necessary. 

• Two readers should be used for presenting a test or portions of a test to 
a student. Using more than one reader helps ensure accuracy of test 
presentation and provides the opportunity for readers to rest during the 
presentation of test material. 

. Students tested through oral reading of the exam must be tested 
individually to prevent the distraction of other students. Moreover, the 
testing of students individually helps ensure that each student receives 
the specific oral reading structure required by his or her specific needs. 

Computer-assisted Testing 

Computer-assisted testing is an accommodation that has received some 
attention through research, though studies concerning its benefit are 
inconclusive (Tindal & Fuchs, 1999). Generally, however, when a student 
uses a computer for daily classroom activities, then this accommodation may 
prove useful during testing if the concepts being tested are not undermined. 

There are several programs and peripheral materials that can be used to 
adapt the computer for use by persons with visual impairments. Screen 
readers, text to speech technology, and accessible keyboard access through 
braille or switches are all available. Depending on the construct being tested, 
test administrators must verify that the student is inhibited from accessing 
software or hardware that may provide an unfair advantage. For example, if 
a student's basic math skills are being assessed and the intent is not to use a 
calculator, then keyboard functions or software used for computations must 
be blocked. For more information on this topic, refer to Test Access: 
Guidelines for Computer-Administered Testing. American Printing House for 
the Blind: Louisville, KY. Available from: 
http://www.aph.org/tests/access/access.pdf 

Response Accommodations 

Students with visual impairments who use the presentation accommodations 
discussed above may also need to use certain response accommodations so 
that answers can be recorded appropriately. As with presentation 
accommodations, response accommodations with which the student is 
familiar are recommended. 

Considerations regarding response accommodations include the following: 

• The student may present answers orally to a test proctor who completes 
the answer sheet. 


72 


Case l:14-cv-00857-TSC Document 70-50 Filed 01/22/16 Page 106 of 127 


• Students may record onto audio tape answers that then must be 
transferred to the answer sheet. 

• The student may need to write answers in the test booklet or on 
separate paper. The student's answers will then need to be transferred 
to the answer sheet. 

• Students may use word processors to write answers that will be 
transferred to the answer sheet. Depending on the construct being 
tested, test administrators must verify that students are inhibited from 
accessing software or hardware that may provide an unfair advantage. 
For example, if a student is responding to a writing prompt and the 
writing will be judged based on correct spelling and grammar, then the 
spell check function and grammar functions must be disabled. 

Each of these accommodations requires a person to transfer the answers 
onto the scanable answer sheet or booklet that will be scored. If computer- 
based testing is used, the transfer of answers is not necessary as this 
process happens as part of the computer test program. The transfer of 
answers must be performed carefully to ensure that the student's answers 
are recorded as intended. 

The following guidelines are provided to ensure that this transfer of 
information is performed appropriately: 

1. Testing materials and the student responses are secure and confidential 
materials, and they must be treated as such to ensure test validity and 
the non-disclosure of the student's identity to unauthorized persons. 

2. Response transcribers must know braille if transcribing braille responses. 

3. Ideally, the response transcriber should be a "neutral" person, not 
someone with a vested interest in the student's scores. 

4. Response transcribers must record the student's use of punctuation, 
spelling, and grammar structure, and provide the student's answer 
exactly as it was delivered by the student. The response transcriber 
cannot record speculative responses for items that the student failed to 
complete. 

5. A second person should be made available to proofread the work of the 
response transcriber in order to ensure that the student's answers have 
been recorded accurately. For the same reason, two transcribers should 
work together in transferring to the answer sheet those graphics that 
the student has produced as an answer to a test item. 

6. For a period of time, student responses must be maintained in a secure 
file with test name, copyright year, form and level administered so that 
the student's actual responses can be reviewed if questions arise. 
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Setting Accommodations 

Frequently, students with visual impairments will need to take a test 
individually or in small groups to ensure that test accommodations are 
implemented without interference to the concentration and test taking of 
other students. If a student is being read to, is recording answers by using 
technology that is noisy, or is recording answers orally, then the student 
must take the test individually and under the supervision of a test 
administrator to prevent the distraction of other test takers. 

The setting for the testing situation must allow space for the materials to be 
used by the student. The manipulation of braille, large print materials, 
braillewriters, talking calculators, and large print materials requires that the 
student be allowed access to a flat, fairly large work area. Moreover, proper 
lighting, while sometimes overlooked, is critical for many readers with visual 
impairments. Lighting that has been adjusted to suit the student's particular 
visual needs will help promote sustained reading efficiency. 

Scheduling Accommodations 

The use of extended time for test completion is a testing accommodation 
that has received considerable attention since state testing and 
accountability systems have been implemented. Research investigating the 
use of extended time has yielded no conclusive information about its benefit 
(Tindal & Fuchs, 1999). However, students with visual impairments will 
usually require extended time during testing because using braille, large 
print, and audio format require more time than does reading standard print 
with acceptable visual acuity. 

A study by Gompel, van Bon, and Schreuder (2004) found that students with 
low vision can read effectively with their low vision aids, using 1 Vi to 2 times 
that needed by regular students. Traditionally, extended time for testing 
large print readers has been 1 Vi time, and for braille readers time allotted 
has been twice as much as that allowed for the standard print reader. 
Another study suggests that experienced braille readers may need no more 
that 50% additional time than the stated duration, with additional time 
allowed for the manipulation of an audio device or the marking of an answer 
sheet (Wetzel & Knowlton 2000). 

Regardless of the time allowed, the student should be carefully monitored to 
ensure that time is being used appropriately. If students need an inordinate 
amount of time, educators may need to investigate the efficiency of the 
chosen reading mode or initiate remediation to improve speed. Generally, 
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timing accommodations should be individualized according to the test taker's 
reading rate and testing situation (Wetzel & Knowlton, 2000). 

Reading braille or large print and listening to material presented orally, 
especially when accompanied by graphic material, can be a fatiguing and 
often frustrating experience in a high stakes testing environment. Therefore, 
students may need several brief sessions in which to take the test. Additional 
break options should also be considered. 

Students may need to be tested at different times of the day depending on 
their optimal functioning time. Students may also need to be tested over a 
longer time period, a week rather than two days, for example. However, any 
alteration of the timetable will necessitate close supervision to ensure test 
security. 


Special Tools Accommodations 

There are a number of special tools that students with visual impairments 
may need during the testing process. Tools provided for sighted students 
during testing, such as calculators, rulers, protractors, or other measurement 
devices, must be provided for students with visual impairments, as well. 
Talking calculators, braille or large print rulers, protractors, and other 
measurement devices do exist, and the student should be allowed to use 
them. When testing allows the use of non-scientific or scientific calculators, 
students with visual impairments should be permitted to use an equivalent 
device that has been adapted for use by the visually impaired user, e.g., a 
non-scientific or scientific talking calculator. Should a state provide 
calculators for the sighted population taking the test, then talking calculators 
should be provided to students with visual impairments who are taking the 
test. Before they are used in a testing situation, electronic and battery- 
operated devices should be inspected to ensure they function properly and 
that the devices contain no saved information, which might provide the user 
an unfair advantage. 

Some other special tools that students with visual impairments might use 
include: 

Abacus: An abacus is often useful for students when mathematics 
problems are to be calculated without a calculator. The abacus functions as 
paper and pencil for some students with visual impairments. 

Graphic Tools: If students are required to produce graphic 
information on a test, they should be allowed to use one of several graphic 
tool kits that exist. It is best if the student uses whatever method he or she 
has used during classroom instruction of graphic construction. The student's 
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constructed graph, if done in braille, will need to be transcribed into print for 
scoring. 

Line Markers and Templates: Occasionally students may want to 
use manipulative devices, such as a ruler or template, to maintain placement 
on a line of braille or print. 

Magnification Devices: Magnification devices include eyeglass- 
mounted magnifiers, free standing or handheld magnifiers, and electronic 
equipment such as the closed circuit television (CCTV) or a computer that 
has text enlargement software installed. These devices do not provide a 
student with an unfair advantage. Rather, they are devices that the student 
requires to access print, and they should be allowed as standard 
accommodations. Should a computer be used as an accommodation, the test 
administrator must ensure that only allowable computer options, such as 
screen enlargement, are used. 

Scientific Tables: Frequently, students may need to refer to a braille 
or large print edition of a scientific table, such as the periodic table of 
elements. 

Physical Manipulatives: Some testing situations may allow that 
objects presented on paper (i.e. money, geometric solids) can be substituted 
with a physical representation of the picture (i.e. penny, nickel, dime, 
quarter, or geometric solids used in instruction). 

Special Paper: Students may need specially designed bold line or 
raised line paper for constructing answers and producing graphs. 

Summary 

This paper has outlined the typical accommodations used by students with 
visual impairments when being tested through use of a written assessment 
such as an academic achievement test. While this discussion is not 
exhaustive of all accommodations that might be used, it is intended to 
provide an understanding of the general accommodations that are expected 
when assessing a student with a visual impairment. Documentation of these 
accommodations on the IEP is crucial as is routine evaluation of their 
effectiveness. 
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APPENDIX G 

Educational Testing Service (ETS) 
GUIDELINES FOR A TEST READER 

The following guidelines will assist in providing the testing 
accommodation of a reader for a test taker with disabilities. If you have 
questions about a specific test, please contact a testing program 
representative. 


Characteristics of a Good Reader 

1. Ability to read aloud clearly, at a normal pace, and with good 
pronunciation. 

2. Familiarity with the words, terms, symbols, or signs that are 
specific to the test content. 

3. Ability to follow instructions to read, verbatim, only the words in 
the test book or on the screen, without changing or adding words 
or assisting the test taker in selecting a response. 

4. Willingness to be patient and to understand that the test taker 
may need to have many test questions repeated several times. 

5. Ability to work with the test taker comfortably and compatibly 
without creating unnecessary pressure or unrealistic expectations. 

General Information for Readers 

1. You must review the test format, subject matter, and sample test 
questions in the testing program's information bulletin or by 
visiting the testing program's Web site. 

2. Prior to beginning the test, you will have the opportunity to meet 
with the test taker, who should be encouraged to discuss matters 
that will affect test performance, e.g., how to determine the 
amount of remaining time and how you can help pace the test taker 
through the test. The opportunity to discuss such questions and 
concerns before the test administration begins will make the test 
administration more effective and fair and will help to minimize 
misunderstandings and misinterpretations. 

3. Test takers who are blind or who have low vision may also have 
special tools or equipment (e.g., abacus, brailler, slate, and stylus) 
that have been approved for use during the test. These tools offer 
neither an unfair nor a special advantage; they are comparable to 
paper and pencil and accomplish the same task. The most 
important consideration is for you and the test taker to have the 
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same set of expectations about what is to happen, how much time 
is allowed, and how all the tasks will be accomplished. 

4. Test takers who are blind or who have low vision may also have 
special tools or equipment (e.g., abacus, brailler, slate, and stylus) 
that have been approved for use during the test. These tools offer 
neither an unfair nor a special advantage; they are comparable to 
paper and pencil and accomplish the same task. The most 
important consideration is for you and the test taker to have the 
same set of expectations about what is to happen, how much time 
is allowed, and how all the tasks will be accomplished. 

5. The test taker may require all or portions of the test to be read 
aloud. The test taker depends on the reader to read the test 
questions accurately, to pronounce words correctly, and to speak 
in a clear voice throughout the test, which may go on for several 
hours. It is a demanding and somewhat tedious task, and not 
everyone is suited to do it. Drinking water should be available for 
you. 

6. Your task is to read only the test questions. Do not try to solve 
problems or determine the correct answer as you read because 
this may result in an unconscious pause or change in inflection 
that could be misleading or disconcerting to the test taker. The 
expression on your face should remain neutral. Do not look at the 
test taker or smile or frown to indicate approval or disapproval. 

7. Read each question as clearly as possible. Give special emphasis 

to words printed in boldface, italics, or capitals, and tell the test- 
taker that the words are printed that way. Do not give your own 
emphasis to words not emphasized in print. 

8. If you find an unfamiliar word or one that you are not sure how to 
pronounce, advise the test taker of your uncertainty about the word 
and spell it. 

9. When reading a word that is pronounced like another word with a 
different spelling, if there can be any doubt about which word is 
intended, spell the word after you have pronounced it. Spell any 
words requested by the test taker. 

10. Avoid getting into conversation about the test questions, but try to 
respond to the test taker's questions by repeating the item, words, 
or instructions as needed. 
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11. When reading passages, be alert to all punctuation marks. Read 
the passage through once so that the test taker can grasp the 
content of the passage. Some test takers may ask for the passage to 
be read through a second time with punctuation marks indicated. 
When required or asked to read, with punctuation, specific lines 
within a passage, indicate all punctuation found within those lines. 

12. When test questions refer to particular lines of a passage, reread 
the lines before reading the question and answer choices. For 
example, you might say, "Question X refers to the following 
lines..." Reading the lines referred to would then be followed by 
reading question X and its response options. 

Special Considerations for Multiple-Choice Tests 

1. Be particularly careful to give equal stress to each response option 
and to read all of them before waiting for a response. The test- 
taker will record the answer or provide the answer to the test 
administrator (writer), who will record it for the test taker. 

2. If you are recording answers and if the test taker designates a 
response choice by letter only ("D", for example), ask if you 
should reread the complete response before the answer is 
recorded. 

3. If the test taker chooses an answer before you have read all the 
answer choices, ask if you should read the other response options. 

4. Allow the test taker to pause before responding. However, if the 
test taker pauses for a considerable time following your reading of 
the answer choices, say: "Do you want me to read the question 
again. ..or any part of it?" In rereading questions, be careful to 
avoid any special emphasis on words not emphasized in the 
printed copy by italics or capitals. 

Mathematics Reading 

A test taker is permitted to ask the reader to write notes and to assist 
with intermediate steps in computing mathematics problems, especially 
if the test taker has no tools or equipment for taking notes or is unable 
to do so. For example, in the multiplication of numbers (e.g., 17 x 521), 
a test taker may say, "Seven times one is seven. Put down the seven. 
Seven twos are fourteen. Put down the four to the left of the seven and 
carry the one." The test taker should be specific in directions to the 
reader as to what he or she writes, in which column to write it, what to 
carry, etc. 

Mathematical expressions must be read precisely and with care to avoid 
misrepresentation for a test taker who has no visual reference. For math 
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items involving algebraic expressions or other mathematical notation, it 
may be preferable for the reader to silently read the entire question 
before reading it aloud to the test taker. Use technically correct yet 
simple terms, and be consistent in the treatment of similar expressions. 
Some typical expressions and the manner in which they should be read 
follow: 

(a) Lowercase letters that are juxtaposed should be read as a 
multiplication expression: 

e.g., xy should be read as "x y," unless it is part of a complex 
expression or this reading is otherwise unclear, in which case read 
it as "x times y." 

(b) Capital and lower-case letters should be differentiated because 
they can have different meanings in mathematical or scientific 
expressions: 

e.g., 


R - 2y = 6 

should be read as "Capital R minus two y equals six." 

(c) Simple numerical fractions should be read as fractions: 
e.g., 


5/6 

Should be read as "five sixths." 

(d) However, similar letter expressions can be read as one letter 
"over" another : 

e.g., 

a 


b 

Should be read as "a over b." 

b+d 


c 

Should be re ad as "a fraction with numerator b plus d and 
denominator c." 

If there is any question as to where the fraction ends, say "end fraction." 

(e) Negative numbers should be read as "negative": 
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e-9-. 


-5 

should be read as "negative five," not "minus five." 


When a subtraction operation is involved, read the sign as "minus, 
e.g. : 


x - 5 

should be read as "x minus five." 

(f) Expressions containing multiple mathematical operations should 
be read exactly as they appear. Expressions containing 
parentheses or brackets can be read in any of the following three 
ways: 


1. quantity, close quantity 

2. paren, close paren (or bracket, close bracket) 

3. left paren, right paren (or left bracket, right bracket) 

For "paren, close paren" or "left paren, right paren," it is also 
acceptable to use "parenthesis" instead of "paren." 

If you use the term "quantity," in complicated expressions, announce 
where enclosed portions end by saying "end quantity:" 

e.g., 


(2x - 6y) - 10 

could be read 

• As "The quantity two x minus six y, close quantity, minus ten;" 

• As "paren, two x minus six y, close paren, minus ten;" 

• Or as "left paren, two x minus six y, right paren, minus ten." 


a (x - y) 


could be read as "a, parenthesis, minus y, close parenthesis." 

a x b 2 
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could be read as "a times the square of b." 

Use pauses to audibly group sections of an expression together: 

e.g., 

z + (-a) 

could be read as "z plus [PAUSE] paren negative a close paren." 

(g) If equations are used in the test you will be reading: 

Since equations are a shorthand means of stating relationships between 
quantities, the reader's job is to translate this shorthand back into 
everyday English. Read equations in this order: 

1. If the equation is numbered, read its number first. 

2. Give the meaning of each letter or symbol 

3. Read the equation. 

e.g.: 

Eq. 6-2 

E = energy in ergs 

m = mass in grams 

c = speed of light in cm. /sec. 

E = me 2 

Read as "Equation six dash two. Capital E equals energy in ergs, m 
equals mass in grams, and c equals the speed of light in 
centimeters per second. Then, Capital E equals m c squared." 

Test Center Procedures for Using a Reader 

1. An approved reader should be admitted to the test center with the 
test taker. The reader's photo-bearing identification should be 
checked. 

2. Prior to the start of the exam, the test center administrator/ 
supervisor will review the Guidelines with the test taker and the 
reader and will set the ground rules for the conduct of the 
examination. 

3. The test administrator must remain in attendance at all times 
during the test administration. 

4. An approved reader is not present to function as an aide to the 
test center staff. It is inappropriate to ask the reader to perform 
clerical duties of any kind. The reader should not be asked to 
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assume any responsibilities belonging to either the center staff or 
the test taker. 

5. Test center staff must ensure that proper test security is maintained 
at all times. It is important that the test administrator ask 
questions and avoid any hasty interpretations of what may be 
communication of test content or exchange of information between 
the test taker and the reader that might give the test-taker an 
unfair advantage. The task requested by the test taker might be 
acceptable once understood. Discussion or communication concerning 
interpretation of test content is not permitted. If such discussion 
occurs and cannot be controlled, or if test center staff observe 
anything they deem unusual, the situation should be reported on the 
Supervisor's Irregularity Report (SIR) or the Electronic Irregularity 
Report (EIR) and the test taker advised of this action. 

6. The test center administrator may also stop the test and dismiss 
the test taker if he or she believes that the reader has provided 
the test taker with any unfair advantage. In such instances, ETS 
reserves the right to cancel the test taker's score. 
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Introduction 

Contracts and Requests for Proposals (RFPs) negotiated between state assessment agencies 
and test publishers carefully outline the responsibilities and expectations for the state 
assessment development and implementation process. While these documents have specific 
points for consideration, often language does not include the assurance of accessible test 
development and implementation. Accessible test items enable all students to participate in the 
assessment process in a way that allows abilities rather than disabilities to be assessed. 
Accessible formats of tests, including the practice tests, must be available for students with visual 
impairments at the same time as their sighted peers. The checklists provided below outline 
considerations for inclusion in each state's RFP or test contract. The usual contractual language 
found in state contracts should be employed, with these special considerations added. 

Universal Design Principles 
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The following guidelines are general considerations for contract and RFP development that 
ensure test development and use for all students, including those with disabilities: 

The same assessment system is used to measure the achievement of all public school 
students in the state. Groups to be included in the state assessment need to be clearly defined. 

The student assessment system provides coherent information on attainment of state 
standards across grades and subjects. 

The tests are designed to be valid and accessible for all students. This includes students with 
disabilities and students with limited English proficiency. 

The tests are aligned with the state's challenging academic content and student achievement 
standards. 

The tests are valid, reliable, technically sound, and consistent with nationally recognized 
professional and technical standards such as national test publisher standards and guidelines 
of the American Psychological Association (APA) and American Educational Research 
Association (AERA). 

The reporting system allows results to be disaggregated (according to the No Child Left Behind 
Act (NCLB) and Individuals with Disabilities Improvement Act of 2004 (IDEA) guidelines) within 
each state and local education agency and school by gender, racial and ethnic group, migrant 
status, disability, socioeconomic status, and limited English proficiency. 

The tests involve multiple up-to-date measures of student academic achievement, including 
measures that assess higher-order thinking skills and understanding. 

The reporting system allows production of individual student reports. 

Roeber, E. (2003). Assessment models for No Child Left Behind. 

Education Commission of the States. http://www.ecs.orq/html/Document.asp?chouseid=4009 

Item Development and Review Process with Publisher 

The following guidelines are provided for consideration as language to include in contracts and 
RFPs that ensure the development and implementation of accessible test formats, specifically for 
students with visual impairments. 

Test publishers must maintain access to experts, i.e. individuals who know and have either 
taught or are knowledgeable about braille, tactile graphics, large print, and audio. These 
individuals can provide information during each phase of test development. 

Experts in visual impairment must be included on Item Writing Committees and Bias Review 
Committees. 


http://www.afb.org/info/afb-national-education-program/jltli-2005-education-summary/checklist-for-rfp-buildin... 
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The use of accommodations must be considered during test item development to ensure 
appropriateness to test purpose and test access. 

The test item pool must be large enough for Bias and Item Review Committees to replace items 
determined to be inaccessible when presented in braille, large print, audio formats, or as tactile 
graphics. 

A representative sample of students with visual impairments needs to be included in any field- 
testing of the assessments, as prescribed in Standard 10.3 (p. 106) of the Standards for 
Educational and Psychological Testing (1999). 

An adequate amount of time for tests and practice tests to proceed through a subcontractor's 
processes needs to be built into contracts so that accessible media as required by each 
student's Individualized Education Program (IEP) are delivered at the same time as the original 
test materials. 

All test administrators' manuals, supplemental manuals which accompany the accessible media 
versions of tests, and local test administrators'/proctors' instructions and training manuals must 
be provided in accessible formats for visually impaired staff. These accessible materials must 
be requested far enough in advance to allow for delivery at the same time as the original test 
materials. 

At the end of each testing season, both students and teachers should give input regarding the 
testing experience. 

Item analyses for accessible format test items will be carried out at the end of each school year 
(or testing season) as part of a continuous improvement plan. 

Allman, C.B. (2004). Test Access: Making tests accessible for students with visual impairments: 

A guide for test publishers, test developers, and state assessment personnel. Second Edition. 

Louisville, KY: American Printing House for the Blind. 

http://www.aph.org/tests/access2/index.html 

Accessible Media Development with Subcontractors 

This section provides guidelines for consideration when contracts are developed with 
subcontractors such as agencies or individuals who will provide tests in one or more accessible 
formats (braille, tactile graphics, large print, and audio). The process may include steps for 
editing, transcribing, designing tactile graphics, proofing, producing and quality checking the 
accessible media. It is essential that the timeline allow adequate time for each of these steps. 

Additional time may need to be built into contracts depending on specific requirements of the 
state such as an independent proofreading by another person or agency, or aligning various 
media for multimedia presentations. 
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The subcontractor must agree to work closely with the test publisher, the state department of 
education, and the test editor. 

The construct to be measured must be documented by the test publisher in test item 
specifications and made available to test editors and accessible media producers. 

Proofreading by a qualified individual, i.e. a person who knows the needed codes and formats 
and is experienced or certified (if applicable), in braille, tactile graphics, large print, and audio 
versions of the test must occur before multiple copies are made. High-stakes tests should be 
proofed a minimum of two times. 

Accessible versions of the test must be aligned so that a multimedia presentation (as approved 
by state assessment programs) is possible if specified by a student's IEP. 

Allowable test format changes, accommodations, and general assistance to test takers by the 
test administrator or proctor must be stated in the test administration manual or supplemental 
materials produced by the subcontractor. 

Subcontractors must be able to meet their deadlines so that high quality accessible media are 
delivered to school systems at the same time as the original test materials. 

Test security and confidentiality standards must be upheld by testing subcontractors. 

Allman, C.B. (2004). Test Access: Making tests accessible for students with visual impairments: 

A guide for test publishers, test developers, and state assessment personnel. Second Edition. 

Louisville, KY: American Printing House for the Blind. 

http://www.aph.org/tests/access2/index.html 

Resources 

Assessment Models for No Child Left Behind, 
from Education Commission of the States (ECS) 
http://www.ecs.org/clearinghouse/40/09/4009.doc 

Building Tests to Support Instruction And Accountability: A Guide for Policymakers, 

from National Education Association (NEA) http://www.nea.org/accountabilitv/buildingtests.html 

Designing School Accountability Systems, 

from Council of Chief State School Officers (CCSSO) 

http://www.nciea.org/publications/desiqSchAccSyst Gonq02.pdf 

Illustrative Language for an RFP to Build Tests to Support Instruction and Accountability, 
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from American Association of School Administrators (AASA) 

http://www.aasa.org/issues and insiqhts/assessment/lllustrative Language for an RFP.pdf 

Information on Writing a Request for Proposal (RFP) 
http://www.arches.uqa.edu/~ninaauq/ITclasses/7550/ 

Model Contractor Standards and State Responsibilities for State Testing Programs, 
from Education Leader's Council (ELC) http://www.accountabilityworks.org/publications 

National Federation of the Blind (NFB/New Hampshire Resolution on Accountability) 
http://www.education-rights.org/brailletwomey1 1 399.html 

Tennessee RFP for Development of Online Tests 
http://www.state.tn.us/finance/rds/ocr/rfp/rfp33104001.pdf 

Test Access: Making Tests Accessible for Students with Visual Impairments: 

A Guide for Test Publishers, Test Developers, and State Assessment Personnel. Second Edition. 
American Printing House for the Blind, http://www.aph.org/tests/access2/index.html 

The Standards for Educational and Psychological Testing (1999), 

American Psychological Association http://www.apa.org/science/standards.html 

Thompson, S., & Thurlow, M. (2002). Universally designed assessments: Better tests for 
everyone! (Policy Directions No. 14). 

Minneapolis, MN: University of Minnesota, National Center on Educational Outcomes. 

Retrieved 1-28-05 from the World Wide Web: 

http://education.umn.edu/NCEO/OnlinePubs/Policy14.htm 

Contributors 

Dr. Carol Allman, Accessible Tests Department with the American Printing House for the Blind, 
allmanc@prodiqy.net 

Barbara Henderson, Accessible Tests Department with the American Printing House for the 
Blind, bhenderson@aph.org 

Debra Sewell, Texas School for the Blind and Visually Impaired, debrasewell@tsbvi.edu 
Mary Ann Siller, American Foundation for the Blind, siller@afb.net 
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Debbie Willis, Accessible Tests Department with the American Printing House for the Blind, 
dwillis@aph.org 

Permission is given to distribute copies with appropriate credit: American Foundation for 
the Blind, American Printing House for the Blind and Texas School for the Blind and 
Visually Impaired from the Josephine L. Taylor Leadership Institute, March 11, 2005. 
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http://www.afb.org/info/afb-national-education-program/jltli-2005-education-summary/checklist-for-rfp-buildin... 



